Page MenuHomePhabricator

MetaMTA - lay some ground work for having an application
ClosedPublic

Authored by btrahan on Jun 22 2015, 8:21 PM.

Details

Summary

Ref T5791. This does a few bits there. Namely:

  • Adds PHID column to PhabricatorMetaMTAMail
    • Implements a PhabricatorMetaMTAMailPHIDType
    • Script to backpopulate them.
  • Makes PhabricatorMetaMTAMail implement PolicyInterface.
    • View policy is NOONE and the author and recipients have automatic view capabilities
    • No edit capability.
  • Adds a PhabricatorMetaMTAMailQuery for PhabricatorMetaMTAMail.
Test Plan

ran ./bin/storage upgrade successfully. commented on a maniphest task and verifed the metamta mail object in the database was created successfully with a shiny new phid

Diff Detail

Repository
rP Phabricator
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

btrahan retitled this revision from to MetaMTA - lay some ground work for having an application.
btrahan updated this object.
btrahan edited the test plan for this revision. (Show Details)
btrahan added a reviewer: epriestley.
btrahan updated this revision to Diff 32429.
epriestley edited edge metadata.Jun 22 2015, 8:27 PM
epriestley accepted this revision.
epriestley added inline comments.
src/applications/metamta/phid/PhabricatorMetaMTAMailPHIDType.php
41

We may want to hold $subject out of this for now, until stuff is a little more mature, although I don't think they're ever actually sensitive.

src/applications/metamta/query/PhabricatorMetaMTAMailQuery.php
20

You may be able to get away with implementing newResultObject(), then just having this do:

return $this->loadStandardPage($this->newResultObject());
62–64

I think you don't need this if you implement getPrimaryTableAlias().

src/applications/metamta/storage/PhabricatorMetaMTAMail.php
1018

You may need to call expandRecipients() on this, too -- I think getAllActorPHIDs() includes recipients like projects in raw form, as PHID-PROJ-xyz. expandRecipients() will expand them into the underlying users, PHID-USER-xyz. This should be moot once edges show up.

This revision is now accepted and ready to land.Jun 22 2015, 8:27 PM
btrahan edited edge metadata.Jun 22 2015, 8:43 PM
btrahan updated this revision to Diff 32432.

updates - thanks!

This revision was automatically updated to reflect the committed changes.

Is it expected that these migrations will take a long time (like > 30 minutes)?

Yeah, I added some guidance on that here:

https://secure.phabricator.com/w/changelog/2015.26/

This period includes mail migrations which may take some time to complete on larger installs.

Not very helpful, but we don't really have much of a way to guess how long things will take because there's huge variance (like, 100x) between installs depending on configuration, hardware, size of the data, etc.

(But I'd be surprised if they took too much longer than ~30 minutes, even on large installs -- I think they took ~8 minutes here, and I recall a couple 15-20 minute reports on fairly active installs.)

One relatively easy thing I think we could do which would often improve the state of the world here is to have MigrationIterator render a progress bar automatically or with some flag. That's less useful diagnostically than emitting output, but in practice the breakdown of issues with migrations is ~100% installs not being able to predict how long they'll take and ~0% them breaking halfway through.

From memory I'm pretty sure this was a migration that took so long for us that I had to cancel it and truncate the mail table so it wouldn't have to migrate data.

For posterity:

  • For rough ballparks, a gzipped sql dump of our db is 2.1 GiB.
  • On our test instance (fork of the prod db from a few weeks ago) running the migration normally took 30-45 minutes.
  • For the production migration we disabled fsync (zfs set sync=disabled tank/lxc/foo) and it took 4m10.926s.