Page MenuHomePhabricator

Implement a stream iterator and hashing iterator
ClosedPublic

Authored by epriestley on Mar 18 2016, 12:05 PM.
Tags
None
Referenced Files
F15416055: D15491.id37342.diff
Thu, Mar 20, 8:57 AM
F15415894: D15491.id37343.diff
Thu, Mar 20, 8:15 AM
F15404807: D15491.id.diff
Tue, Mar 18, 9:11 AM
F15399585: D15491.diff
Mon, Mar 17, 5:20 AM
F15382706: D15491.diff
Fri, Mar 14, 2:00 PM
F15374946: D15491.id37343.diff
Wed, Mar 12, 8:49 PM
F15334090: D15491.id37342.diff
Sat, Mar 8, 6:44 AM
Unknown Object (File)
Thu, Feb 27, 8:39 AM
Subscribers
None

Details

Summary

Ref T10604. For Git LFS, we need to compute file hashes as we accept data.

This allows a stream to be wrapped in an iterator, and an iterator to be wrapped in another iterator which does hashing.

This will ultimately let us pump data into Files without needing 100GB of memory to accept a 100GB file.

Test Plan

Added unit test, ran unit test.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Implement a stream iterator and hashing iterator.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley edited edge metadata.
  • Improve behavior of next() on an already-invalid stream.
chad edited edge metadata.
This revision is now accepted and ready to land.Mar 18 2016, 3:49 PM
This revision was automatically updated to reflect the committed changes.