Page MenuHomePhabricator

In "Download Raw Diff", engage the chunk engine to handle 8MB+ changes
ClosedPublic

Authored by epriestley on Jun 17 2019, 4:51 PM.
Tags
None
Referenced Files
F13086896: D20579.diff
Thu, Apr 25, 12:46 AM
Unknown Object (File)
Mar 23 2024, 12:42 PM
Unknown Object (File)
Mar 23 2024, 12:42 PM
Unknown Object (File)
Mar 16 2024, 12:46 AM
Unknown Object (File)
Feb 21 2024, 4:10 AM
Unknown Object (File)
Feb 19 2024, 10:24 AM
Unknown Object (File)
Feb 14 2024, 6:51 AM
Unknown Object (File)
Feb 2 2024, 1:41 PM
Subscribers
None

Details

Summary

Fixes T13313. The "Download Raw Diff" workflow in Differential currently uses an older way of interacting with Files that doesn't engage the chunk engine and can't handle 8MB+ files.

Update to IteratorFileUploadSource -- we're still passing in a single giant blob, but this approach can be chunked.

This will still break somewhere north of 8MB (it will break at 2GB with the PHP string limit if nowhere sooner, since we're putting the entire raw diff in $raw_diff rather than using a rope/stream) but will likely survive diffs in the hundreds-of-megabytes range for now.

Test Plan
  • Added str_repeat('x', 1024 * 1024 * 9) to the $raw_diff to create a 9MB+ diff.
  • Configured file storage with no engine explicitly configured for >8MB chunks (i.e., "reasonably").
  • Clicked "Download Raw Diff".
  • Before: misleading file storage engine error ("no engine can store this file").
  • After: large, raw diff response.

Diff Detail

Repository
rP Phabricator
Branch
file1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 22959
Build 31508: Run Core Tests
Build 31507: arc lint + arc unit