Page MenuHomePhabricator

Modernize file storage engine selection
ClosedPublic

Authored by epriestley on Mar 12 2015, 7:39 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Dec 13, 12:20 PM
Unknown Object (File)
Fri, Nov 29, 7:12 AM
Unknown Object (File)
Fri, Nov 29, 7:12 AM
Unknown Object (File)
Fri, Nov 29, 6:23 AM
Unknown Object (File)
Fri, Nov 29, 12:40 AM
Unknown Object (File)
Fri, Nov 29, 12:40 AM
Unknown Object (File)
Fri, Nov 29, 12:39 AM
Unknown Object (File)
Fri, Nov 29, 12:38 AM
Subscribers

Details

Summary

Fixes T5843. File storage engines use a very old "selector" mechanism which makes them difficult to extend.

This mechanism predates widespread use of PhutilSymbolLoader to discover available implementations at runtime. Runtime discovery has generally proven more flexible and easier to use than explicit selection (although it sometimes needs more UI to support it in cases where order or enabled/disabled flags can not be directly determined).

Use a modern runtime discovery mechanism instead of an explicit selector. This might break any installs which subclassed the Selector, but I believe almost no such installs exist, and they'll receive a meaningful exception upon upgrading (any custom engines will no longer implement all of the required methods).

Looking forward, this modernizes infrastructure to prepare for new "virtual" chunked-storage engines, with the eventual goal of supporting very large file uploads and data import into the Phacility cluster.

This uses D12051 to add UI to make it easier to understand the state of storage engines.

Test Plan

Used new UI panel to assess storage engines:

Screen_Shot_2015-03-12_at_12.28.29_PM.png (195×1 px, 37 KB)

  • Uploaded a small file, saw it go to MySQL engine.
  • Uploaded a larger file, saw it go to S3 engine.

Diff Detail

Repository
rP Phabricator
Branch
sengine2
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 4857
Build 4874: [Placeholder Plan] Wait for 30 Seconds

Event Timeline

epriestley retitled this revision from to Modernize file storage engine selection.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: btrahan.
  • Better error message if no engines are avialable.

I'd like to move all the storage.blah config keys into this UI panel eventually, too, but don't plan to do that anytime soon.

btrahan edited edge metadata.
This revision is now accepted and ready to land.Mar 12 2015, 8:03 PM
This revision was automatically updated to reflect the committed changes.