Page MenuHomePhabricator

MetaMTA - lay some ground work for having an application

Authored by btrahan on Jun 22 2015, 8:21 PM.



Ref T5791. This does a few bits there. Namely:

  • Adds PHID column to PhabricatorMetaMTAMail
    • Implements a PhabricatorMetaMTAMailPHIDType
    • Script to backpopulate them.
  • Makes PhabricatorMetaMTAMail implement PolicyInterface.
    • View policy is NOONE and the author and recipients have automatic view capabilities
    • No edit capability.
  • Adds a PhabricatorMetaMTAMailQuery for PhabricatorMetaMTAMail.
Test Plan

ran ./bin/storage upgrade successfully. commented on a maniphest task and verifed the metamta mail object in the database was created successfully with a shiny new phid

Diff Detail

rP Phabricator
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

btrahan updated this revision to Diff 32429.Jun 22 2015, 8:21 PM
btrahan retitled this revision from to MetaMTA - lay some ground work for having an application.
btrahan updated this object.
btrahan edited the test plan for this revision. (Show Details)
btrahan added a reviewer: epriestley.
epriestley accepted this revision.Jun 22 2015, 8:27 PM
epriestley edited edge metadata.
epriestley added inline comments.

We may want to hold $subject out of this for now, until stuff is a little more mature, although I don't think they're ever actually sensitive.


You may be able to get away with implementing newResultObject(), then just having this do:

return $this->loadStandardPage($this->newResultObject());

I think you don't need this if you implement getPrimaryTableAlias().


You may need to call expandRecipients() on this, too -- I think getAllActorPHIDs() includes recipients like projects in raw form, as PHID-PROJ-xyz. expandRecipients() will expand them into the underlying users, PHID-USER-xyz. This should be moot once edges show up.

This revision is now accepted and ready to land.Jun 22 2015, 8:27 PM
btrahan updated this revision to Diff 32432.Jun 22 2015, 8:43 PM
btrahan edited edge metadata.

updates - thanks!

This revision was automatically updated to reflect the committed changes.

Is it expected that these migrations will take a long time (like > 30 minutes)?

Yeah, I added some guidance on that here:

This period includes mail migrations which may take some time to complete on larger installs.

Not very helpful, but we don't really have much of a way to guess how long things will take because there's huge variance (like, 100x) between installs depending on configuration, hardware, size of the data, etc.

(But I'd be surprised if they took too much longer than ~30 minutes, even on large installs -- I think they took ~8 minutes here, and I recall a couple 15-20 minute reports on fairly active installs.)

One relatively easy thing I think we could do which would often improve the state of the world here is to have MigrationIterator render a progress bar automatically or with some flag. That's less useful diagnostically than emitting output, but in practice the breakdown of issues with migrations is ~100% installs not being able to predict how long they'll take and ~0% them breaking halfway through.

From memory I'm pretty sure this was a migration that took so long for us that I had to cancel it and truncate the mail table so it wouldn't have to migrate data.

For posterity:

  • For rough ballparks, a gzipped sql dump of our db is 2.1 GiB.
  • On our test instance (fork of the prod db from a few weeks ago) running the migration normally took 30-45 minutes.
  • For the production migration we disabled fsync (zfs set sync=disabled tank/lxc/foo) and it took 4m10.926s.