Page MenuHomePhabricator

Improve workflow for host-to-host Phacility cluster repository migrations of observed repositories
Closed, ResolvedPublic

Description

See Migrating Repository Shards. Currently, migrating repositories from host to host goes through a few steps:

  • Mark the repository as read-only.
  • Snapshot and upload the repository from the old host.
  • Download the repository on the new host.
  • Update the configuration.
  • Mark the repository as read/write.

This is good (and approximately minimal) in the general case, but more complicated than necessary when a repository is observed, since observed repositories are never writable anyway. Observed repositories can be migrated like this:

  1. (Optionally) Fetch the repository on the new host.
  2. Update the configuration.

Step (1) isn't technically necessary, but can help minimize temporary impacts (e.g., the web UI failing until the fetch completes).

This sequence can be accomplished by skipping the bin/services save-repository step and modifying bin/services load-repository slightly so it can perform a fetch instead of a download.

This also requires one minor change to bin/repository pull, which currently refuses to pull a repository that isn't local to the host according to cluster configuration. In this workflow, we're explicitly trying to violate cluster locality rules (since we plan to change them after the fetch completes).

So the approach here is:

  • Add a --ignore-locality flag to bin/repository pull.
  • Add an --observe flag to bin/services load-repository that fails if the repository isn't observed, then runs bin/repository pull --ignore-locality ... instead of downloading and extracting the repository.
  • Update the documentation.

Revisions and Commits

Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
rP Phabricator
D21544
D21544

Event Timeline

epriestley added a revision: Restricted Differential Revision.Feb 4 2021, 6:29 PM

This more-or-less worked as desired. Some minor issues:

  • Observed repositories with broken remotes can't take advantage of this, but that's probably correct.
  • Currently, neither flow correctly handles repositories with no data (like repositories that were never activated). You can work around this by creating an empty data directory, then using the general workflow.
epriestley added a commit: Restricted Diffusion Commit.Feb 7 2021, 6:36 PM