Page MenuHomePhabricator

Implement a stream iterator and hashing iterator
ClosedPublic

Authored by epriestley on Mar 18 2016, 12:05 PM.
Tags
None
Referenced Files
F13059487: D15491.diff
Fri, Apr 19, 4:20 PM
Unknown Object (File)
Tue, Apr 9, 3:43 AM
Unknown Object (File)
Mon, Apr 8, 6:50 AM
Unknown Object (File)
Feb 24 2024, 4:50 PM
Unknown Object (File)
Feb 24 2024, 7:50 AM
Unknown Object (File)
Feb 3 2024, 9:14 AM
Unknown Object (File)
Jan 24 2024, 10:38 PM
Unknown Object (File)
Jan 15 2024, 1:26 AM
Subscribers
None

Details

Summary

Ref T10604. For Git LFS, we need to compute file hashes as we accept data.

This allows a stream to be wrapped in an iterator, and an iterator to be wrapped in another iterator which does hashing.

This will ultimately let us pump data into Files without needing 100GB of memory to accept a 100GB file.

Test Plan

Added unit test, ran unit test.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Implement a stream iterator and hashing iterator.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley edited edge metadata.
  • Improve behavior of next() on an already-invalid stream.
chad edited edge metadata.
This revision is now accepted and ready to land.Mar 18 2016, 3:49 PM
This revision was automatically updated to reflect the committed changes.