Improve performance of "phabricator:20210215.changeset.02.phid-populate.php"
ClosedPublic
Actions

Authored by epriestley on Feb 19 2021, 3:48 PM.

Details

Reviewers: None
Maniphest Tasks: T13613: Improve performance of bulk PHID assignment, particularly for "phabricator:20210215.changeset.02.phid-populate.php"
Commits: rPb3976acc40a6: Improve performance of "phabricator:20210215.changeset.02.phid-populate.php"

Summary

Ref T13613. Improve the performance of this migration by using a temporary table and an "UPDATE x JOIN y ..." pattern.

Test Plan

Ran on secure, got exit after a few seconds since the migration is idempotent and changesets already had PHIDs.
Ran on secure with the continue; commented out, got valid new PHIDs in 53s (from 153s).
Tried a larger page size (16K), didn't see any improvement.
From "--trace", client PHID generation seems to be the limiting factor.

Diff Detail

Repository

rP Phabricator

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

epriestley created this revision.Feb 19 2021, 3:48 PM

Owners added a subscriber: Restricted Owners Package.Feb 19 2021, 3:48 PM

Harbormaster completed remote builds in B25152: Diff 51343.Feb 19 2021, 3:49 PM

epriestley requested review of this revision.Feb 19 2021, 3:49 PM

From "--trace", client PHID generation seems to be the limiting factor.

I think this isn't quite true. On secure, generating 400K PHIDs in a standalone script takes 7s, so this isn't totally trivial but is only ~13% of remaining runtime. I don't see any obvious wins remaining here, though.

This revision was not accepted when it landed; it landed in state Needs Review.Feb 19 2021, 3:53 PM

Closed by commit rPb3976acc40a6: Improve performance of "phabricator:20210215.changeset.02.phid-populate.php" (authored by epriestley). · Explain Why

This revision was automatically updated to reflect the committed changes.

epriestley added a commit: rPb3976acc40a6: Improve performance of "phabricator:20210215.changeset.02.phid-populate.php".