Page MenuHomePhabricator

Reindex dashboards and panels (allow migrations to queue a job to queue other indexing jobs)
ClosedPublic

Authored by epriestley on Apr 12 2019, 9:27 PM.

Details

Summary

Depends on D20411. Ref T13272. Dashboards and panels have new indexes (Ferret and usage edges) that need a rebuild.

For large datasets like commits we have the "activity" flow in T11932, but realistically these rebuilds won't take more than a few minutes on any realistic install so we should be able to just queue them up as migrations.

Let migrations insert a job to basically run bin/search index --type SomeObjectType, then do that for dashboards and panels.

(I'll do Herald rules in a followup too, but I want to tweak one indexing thing there.)

Test Plan

Ran the migration, ran bin/phd debug task, saw everything get indexed with no manual intervention.

Diff Detail

Repository
rP Phabricator
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

epriestley created this revision.Apr 12 2019, 9:27 PM
Owners added a subscriber: Restricted Owners Package.Apr 12 2019, 9:27 PM
Harbormaster returned this revision to the author for changes because remote builds failed.
  • Make the worker unit tests truncate the table before they run; otherwise, they get confused by the tasks inserted by these migrations.
epriestley requested review of this revision.Apr 12 2019, 11:02 PM
amckinley accepted this revision.Apr 17 2019, 5:37 PM
amckinley added inline comments.
resources/sql/autopatches/20190412.dashboard.13.rebuild.php
4–8

Maybe put these in a try/catch so users don't end up with a failed migration if this doesn't work for some reason?

This revision is now accepted and ready to land.Apr 17 2019, 5:37 PM
epriestley added inline comments.Apr 17 2019, 7:05 PM
resources/sql/autopatches/20190412.dashboard.13.rebuild.php
4–8

If there's some reason this may fail I'd like to learn about it and fix it, ideally. If someone reports something dumb that we can't really fix (or detect?) we could try/catch our way around it, but these shouldn't fail for any legitimate reason I'm aware of.

This revision was automatically updated to reflect the committed changes.

(And the reindex is safe/idempotent, so some reasonable failures like "disk full" would be bad to try/catch our way through, since the correct remedy is to free some space and retry the migration.)

Or "db002, which has the worker database, is being super flaky, even though db001 with every other database is up" or something, although that may not actually lead to a possible state where the INSERT can fail and the patch can still be marked as successful.