Page MenuHomePhabricator

Upgrading: Differential Hunk Migration
Closed, ResolvedPublic

Description

Differential stores the source text for code reviews in a "hunk" table. This table has two formats, a "legacy" format and a "modern" format. The modern format is better, and supports compression and other features.

New hunks have been written to the "modern" table for a very long time (perhaps a year or more?) but installs which have been around for longer than that may still have some old hunks stored in the "legacy" format. In a few weeks years, we will add a mandatory hunk format migration which migrates any legacy hunks to modern hunk storage.

For installs with a long history, this may take some time. This migration is safe to run online (while Phabricator is running), and you can potentially avoid maintenance downtime by running this migration manually in advance of the forced migration.

Determining If You Are Affected

To determine if you are affected, run bin/storage probe and look at the size of the differential_hunk table in the phabricator_differential database.

  • If this table has size 0.0 MB, your install has no legacy hunks. You are not affected at all and can ignore everything happening here.
  • If this table has a small size (up to about 100 MB), your install has some legacy hunks but the migration should happen fairly quickly (a few minutes). You can run the migration manually if you prefer, or wait for the forced migration.
  • If this table is large (bigger than 100 MB) you may be able to reduce maintenance downtime significantly by running the migration manually in advance of the forced migration.

Manual Migration

To run the migration manually, run this command after upgrading to HEAD of master or stable (2015 Week 26 or later):

phabricator/ $ ./bin/hunks migrate

This command can safely be run while Phabricator is online and serving normal traffic.

Context and Discussion

See T8475 and linked objects for technical context and discussion.

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added projects: Guides, Differential.
epriestley added subscribers: epriestley, joshuaspence.

I marked D13222 as closing this as I figured that when D13222 lands this will no longer be relevant.

epriestley claimed this task.

This has now been live for a while without anyone hitting real issues other than the minor hiccup in T12090. T8475 is an eventual followup.