Page MenuHomePhabricator

Support storage of Differential hunk data in Files

Authored by epriestley on Sep 11 2017, 3:31 PM.
Referenced Files
F12186071: D18584.diff
Sat, Sep 16, 4:23 PM
F12182888: D18584.id44628.diff
Wed, Sep 13, 11:22 PM
F12167263: D18584.diff
Thu, Sep 7, 2:38 AM
F12156443: D18584.diff
Mon, Sep 4, 11:10 PM
F12133897: D18584.diff
Aug 25 2023, 2:06 AM
F12130301: D18584.id44630.diff
Aug 22 2023, 7:20 PM
Unknown Object (File)
Aug 17 2023, 8:12 AM
Unknown Object (File)
Jul 20 2023, 12:30 AM



Ref T12932. For long-lived installs, one of the largest tables tends to be the hunk data table. Although it doesn't grow tremendously fast, it's also well suited to storage in Files instead of the database (infrequent access, relatively large blobs of data, mostly one-at-a-time access), and earlier work anticipated eventually adding support for Files storage.

Make Files storage work, and provide bin/differential migrate-hunk to manually test/migrate hunks. This is currently the only way hunks get moved to file storage, but I expect to add a GC step which moves them to File storage after 30 days shortly.

The immediate motivation for this is to relieve storage pressure on db001/db002 so we have more headroom for deploying the Ferret engine and its larger indexes (see also T12819).

Test Plan
  • Used bin/differential migrate-hunk to move a hunk to and from file storage, verified it survived intact.
  • Downloaded the actual stored file, sanity-checked it. Verified permissions.
  • Destroyed a diff with bin/remove destroy, saw the hunk and file storage destroyed.
  • Verified that going from file -> text destroys the old file properly with migrate-hunk --trace ....

Diff Detail

rP Phabricator
Lint Not Applicable
Tests Not Applicable