Page MenuHomePhabricator

Implement a stream iterator and hashing iterator
ClosedPublic

Authored by epriestley on Mar 18 2016, 12:05 PM.
Tags
None
Referenced Files
F15570759: D15491.id37350.diff
Sun, May 4, 9:52 PM
F15569664: D15491.id37342.diff
Sun, May 4, 3:28 AM
F15569578: D15491.id37342.diff
Sun, May 4, 2:02 AM
F15562007: D15491.id37343.diff
Tue, Apr 29, 11:49 PM
F15538369: D15491.id37342.diff
Thu, Apr 24, 6:49 PM
F15535936: D15491.id.diff
Thu, Apr 24, 7:42 AM
F15507133: D15491.diff
Tue, Apr 15, 4:40 PM
F15482238: D15491.id37342.diff
Apr 9 2025, 3:47 AM
Subscribers
None

Details

Summary

Ref T10604. For Git LFS, we need to compute file hashes as we accept data.

This allows a stream to be wrapped in an iterator, and an iterator to be wrapped in another iterator which does hashing.

This will ultimately let us pump data into Files without needing 100GB of memory to accept a 100GB file.

Test Plan

Added unit test, ran unit test.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Implement a stream iterator and hashing iterator.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley edited edge metadata.
  • Improve behavior of next() on an already-invalid stream.
chad edited edge metadata.
This revision is now accepted and ready to land.Mar 18 2016, 3:49 PM
This revision was automatically updated to reflect the committed changes.