Page MenuHomePhabricator

Implement a stream iterator and hashing iterator
ClosedPublic

Authored by epriestley on Mar 18 2016, 12:05 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Nov 24, 4:52 PM
Unknown Object (File)
Tue, Nov 19, 10:11 PM
Unknown Object (File)
Fri, Nov 15, 6:49 PM
Unknown Object (File)
Mon, Nov 11, 7:51 PM
Unknown Object (File)
Sun, Nov 10, 7:38 PM
Unknown Object (File)
Fri, Nov 8, 11:13 AM
Unknown Object (File)
Oct 26 2024, 12:20 AM
Unknown Object (File)
Oct 23 2024, 2:13 PM
Subscribers
None

Details

Summary

Ref T10604. For Git LFS, we need to compute file hashes as we accept data.

This allows a stream to be wrapped in an iterator, and an iterator to be wrapped in another iterator which does hashing.

This will ultimately let us pump data into Files without needing 100GB of memory to accept a 100GB file.

Test Plan

Added unit test, ran unit test.

Diff Detail

Repository
rPHU libphutil
Branch
lfs11
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 11202
Build 13901: Run Core Tests
Build 13900: arc lint + arc unit

Event Timeline

epriestley retitled this revision from to Implement a stream iterator and hashing iterator.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley edited edge metadata.
  • Improve behavior of next() on an already-invalid stream.
chad edited edge metadata.
This revision is now accepted and ready to land.Mar 18 2016, 3:49 PM
This revision was automatically updated to reflect the committed changes.