Page MenuHomePhabricator
Differential D15491

Implement a stream iterator and hashing iterator
ClosedPublic
Actions

Authored by epriestley on Mar 18 2016, 12:05 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Apr 29, 2:42 PM
Unknown Object (File)
Wed, Apr 24, 10:44 PM
Unknown Object (File)
Wed, Apr 24, 6:24 PM
Unknown Object (File)
Tue, Apr 23, 2:27 AM
Unknown Object (File)
Fri, Apr 19, 4:20 PM
Unknown Object (File)
Tue, Apr 9, 3:43 AM
Unknown Object (File)
Mon, Apr 8, 6:50 AM
Unknown Object (File)
Feb 24 2024, 4:50 PM
View All Files
Subscribers
None

Details

Reviewers
chad
Maniphest Tasks
T10604: Allow application code to stream request data
Commits
rPHU84b0feab4220: (stable) Promote 2016 Week 12
rPHUb4f38af38480: Implement a stream iterator and hashing iterator
Summary

Ref T10604. For Git LFS, we need to compute file hashes as we accept data.

This allows a stream to be wrapped in an iterator, and an iterator to be wrapped in another iterator which does hashing.

This will ultimately let us pump data into Files without needing 100GB of memory to accept a 100GB file.

Test Plan

Added unit test, ran unit test.

Diff Detail

Repository
rPHU libphutil
Branch
lfs11
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 11202
Build 13901: Run Core Tests
Build 13900: arc lint + arc unit

Event Timeline

epriestley updated this revision to Diff 37342.Mar 18 2016, 12:05 PM
epriestley retitled this revision from to Implement a stream iterator and hashing iterator.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley added a task: T10604: Allow application code to stream request data.
Herald edited edge metadata. · View Herald TranscriptMar 18 2016, 12:05 PM
epriestley updated this revision to Diff 37343.Mar 18 2016, 1:22 PM
epriestley edited edge metadata.
  • Improve behavior of next() on an already-invalid stream.
Herald edited edge metadata. · View Herald TranscriptMar 18 2016, 1:22 PM
epriestley mentioned this in T7789: Support Git Large File Storage.Mar 18 2016, 1:42 PM
chad accepted this revision.Mar 18 2016, 3:49 PM
chad edited edge metadata.
This revision is now accepted and ready to land.Mar 18 2016, 3:49 PM
Closed by commit rPHUb4f38af38480: Implement a stream iterator and hashing iterator (authored by epriestley, committed by epriestley). · Explain WhyMar 18 2016, 4:11 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
src/
17 lines
aphront/
requeststream/
34 lines
utils/
40 lines
57 lines
64 lines
__tests__/
45 lines
hashingiterator/
1007 lines

Diff 37342

View Options

src/__phutil_library_map__.php

Loading...
View Options

src/aphront/requeststream/AphrontRequestStream.php

Loading...
View Options

src/utils/PhutilHashingIterator.php

Loading...
View Options

src/utils/PhutilProxyIterator.php

Loading...
View Options

src/utils/PhutilStreamIterator.php

Loading...
View Options

src/utils/__tests__/PhutilHashingIteratorTestCase.php

Loading...
View Options

src/utils/__tests__/hashingiterator/mostlyprime.txt

Loading...
Phabricator · Built by Phacility