Page MenuHomePhabricator
Create Task
Maniphest T11978

Storage Tools Not Actually Upgrading Database
Closed, ResolvedPublic
Actions

Description

Set up a new install relatively recently on stable. When the storage upgrade is executed on the database when pulling down new source releases, it appears that the storage tool successfully reads current database state but does not actually apply changes to the database. The database is accessible and the tool can connect without issue; for instance, storage adjust can find something like a missing key e.g. phabricator_search.search_documentfield.key_corpus, however when attempting to apply the adjustment it fails. At this point in time the last three autopatches have not been executed against the target database, but are marked as applied via storage status. Additional info:

  • Uses root with MySQL
  • All web-based functions work without issue (until the attempted upgrade which happened to break the whole site due to a now-missing table)
  • Patches have been marked as applied, but have not been
  • No errors are logged when the upgrade first runs, specifically bin/storage upgrade --force
  • The tool presents Storage is up to date and then when Synchronizing static tables... runs it emits an error such as Table 'phabricator_search.stopwords' doesn't exist

Can a patch be re-executed, even if marked as applied? Would help in getting additional information.

Could definitely be an environment setup issue but not seeing why the tool would partially act like things are successful yet not actually apply the patch to MySQL.

Event Timeline

mbishopim3 created this task.Dec 8 2016, 10:38 PM
epriestley added a subscriber: epriestley.Dec 8 2016, 10:54 PM

You can use bin/storage upgrade --apply phabricator:2016... to reapply a specific patch explicitly, even if it has previously been marked as applied. You can get the full names of patches from bin/storage status.

chad edited projects, added Bug Report (Needs Information); removed Bug Report.Dec 8 2016, 11:09 PM
mbishopim3 added a comment.Dec 9 2016, 2:11 PM

Re-applied the scripts and all is well now. Once another SQL script comes into the stable branch I will run the upgrade again, something must still be off here. For reference, after a pull, daemon stop, etc. we are only running storage upgrade --force -- is this incorrect? Seems to be fine in our other instances, but for this one the scripts are being marked as applied, but they aren't.

epriestley added a comment.Dec 9 2016, 2:14 PM

That's correct. I'm not sure what's going wrong. My guesses are the crazy stuff you might expect like:

  • You have several different configs, and are selecting different CLI and web configs?
  • You're using --user or --host flags or similar?
  • You've configured cluster.databases in a weird way I didn't expect?

But you'd presumably have mentioned that kind of thing.

(It's also possible this issue is more widespread and we just haven't made too many important schema changes since whatever broke it was introduced, but I'd expect to see a lot of reports if that was the case.)

mbishopim3 added a comment.Jan 4 2017, 3:34 PM

This can be closed as invalid (or resolved I guess). Looks like we performed an install and some upgrades at an awkward time when some storage upgrade changes were occurring. We've run several stable branch updates on the install since and it's been fine.

epriestley closed this task as Resolved.Jan 4 2017, 3:37 PM
epriestley claimed this task.

Cool, glad things are back in working order. Thanks for following up.

Phabricator · Built by Phacility