Page MenuHomePhabricator

Old migration resizes and writes new Conpherence room images, but Files have changed since then
Closed, ResolvedPublic

Description

Situation:

  • Phabricator on Ubuntu v14 with MySql
  • Upgraded Ubuntu to Ubuntu v16
  • Git pull on all 3 repositories
  • Phabricator didn't start anymore. I Assumed it was PHP 7.0. Downgraded to PHP 5.6, but still got the error:
  • (The /version page does not work, so version information is provided manually)
notroot@xxx:~$ php -v
PHP 5.6.30-10+deb.sury.org~xenial+2 (cli)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
notroot@xxx:~$ apache2 -v
Server version: Apache/2.4.18 (Ubuntu)
Server built:   2016-07-14T12:32:26
notroot@xxx:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial
+ /var/www/php/phabricator/phabricator/bin/storage upgrade --force
Applying patch "phabricator:20161005.conpherence.image.2.php" to host "localhost
"...
[2017-04-23 07:10:06] EXCEPTION: (AphrontSchemaQueryException) #1146: Table 'phabricator_auth.auth_hmackey' doesn't exist at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:325]
arcanist(head=master, ref.master=146693307f60), phabricator(head=master, ref.master=bc9291b32708), phutil(head=master, ref.master=6fe33623cda6)
  #0 <#2> AphrontBaseMySQLDatabaseConnection::throwCommonException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:348]
  #1 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryCodeException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:289]
  #2 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryException(mysqli) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:185]
  #3 <#2> AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [<phutil>/src/xsprintf/queryfx.php:8]
  #4 <#2> queryfx(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:13]
  #5 <#2> queryfx_all(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:19]
  #6 <#2> queryfx_one(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:220]
  #7 <#2> PhabricatorHash::readHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:177]
  #8 <#2> PhabricatorHash::getNamedHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:146]
  #9 <#2> PhabricatorHash::digestWithNamedKey(string, string) called at [<phabricator>/src/applications/files/engine/PhabricatorFileStorageEngine.php:374]
  #10 <#2> PhabricatorFileStorageEngine::newIntegrityHash(string, PhabricatorFileRawStorageFormat) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:520]
  #11 <#2> PhabricatorFile::writeToEngine(PhabricatorMySQLFileStorageEngine, string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:351]
  #12 phlog(AphrontSchemaQueryException) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:366]
  #13 PhabricatorFile::buildFromFileData(string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:419]
  #14 PhabricatorFile::newFromFileData(string, array) called at [<phabricator>/src/applications/files/transform/PhabricatorFileImageTransform.php:151]
  #15 PhabricatorFileImageTransform::newFileFromData(string) called at [<phabricator>/src/applications/files/transform/PhabricatorFileThumbnailTransform.php:228]
  #16 PhabricatorFileThumbnailTransform::getDefaultTransform(PhabricatorFile) called at [<phabricator>/src/applications/files/transform/PhabricatorFileTransform.php:27]
  #17 PhabricatorFileTransform::executeTransform(PhabricatorFile) called at [<phabricator>/resources/sql/autopatches/20161005.conpherence.image.2.php:25]
  #18 require_once(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:285]
  #19 PhabricatorStorageManagementAPI::applyPatchPHP(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:241]
  #20 PhabricatorStorageManagementAPI::applyPatch(PhabricatorStoragePatch) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:1093]
  #21 PhabricatorStorageManagementWorkflow::doUpgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:840]
  #22 PhabricatorStorageManagementWorkflow::upgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementUpgradeWorkflow.php:78]
  #23 PhabricatorStorageManagementUpgradeWorkflow::didExecute(PhutilArgumentParser) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:107]
  #24 PhabricatorStorageManagementWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:441]
  #25 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:333]
  #26 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/sql/manage_storage.php:249]
[2017-04-23 07:10:06] EXCEPTION: (AphrontSchemaQueryException) #1146: Table 'phabricator_auth.auth_hmackey' doesn't exist at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:325]
arcanist(head=master, ref.master=146693307f60), phabricator(head=master, ref.master=bc9291b32708), phutil(head=master, ref.master=6fe33623cda6)
  #0 <#2> AphrontBaseMySQLDatabaseConnection::throwCommonException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:348]
  #1 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryCodeException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:289]
  #2 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryException(mysqli) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:185]
  #3 <#2> AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [<phutil>/src/xsprintf/queryfx.php:8]
  #4 <#2> queryfx(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:13]
  #5 <#2> queryfx_all(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:19]
  #6 <#2> queryfx_one(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:220]
  #7 <#2> PhabricatorHash::readHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:177]
  #8 <#2> PhabricatorHash::getNamedHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:146]
  #9 <#2> PhabricatorHash::digestWithNamedKey(string, string) called at [<phabricator>/src/applications/files/engine/PhabricatorFileStorageEngine.php:374]
  #10 <#2> PhabricatorFileStorageEngine::newIntegrityHash(string, PhabricatorFileRawStorageFormat) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:520]
  #11 <#2> PhabricatorFile::writeToEngine(PhabricatorLocalDiskFileStorageEngine, string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:351]
  #12 phlog(AphrontSchemaQueryException) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:366]
  #13 PhabricatorFile::buildFromFileData(string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:419]
  #14 PhabricatorFile::newFromFileData(string, array) called at [<phabricator>/src/applications/files/transform/PhabricatorFileImageTransform.php:151]
  #15 PhabricatorFileImageTransform::newFileFromData(string) called at [<phabricator>/src/applications/files/transform/PhabricatorFileThumbnailTransform.php:228]
  #16 PhabricatorFileThumbnailTransform::getDefaultTransform(PhabricatorFile) called at [<phabricator>/src/applications/files/transform/PhabricatorFileTransform.php:27]
  #17 PhabricatorFileTransform::executeTransform(PhabricatorFile) called at [<phabricator>/resources/sql/autopatches/20161005.conpherence.image.2.php:25]
  #18 require_once(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:285]
  #19 PhabricatorStorageManagementAPI::applyPatchPHP(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:241]
  #20 PhabricatorStorageManagementAPI::applyPatch(PhabricatorStoragePatch) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:1093]
  #21 PhabricatorStorageManagementWorkflow::doUpgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:840]
  #22 PhabricatorStorageManagementWorkflow::upgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementUpgradeWorkflow.php:78]
  #23 PhabricatorStorageManagementUpgradeWorkflow::didExecute(PhutilArgumentParser) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:107]
  #24 PhabricatorStorageManagementWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:441]
  #25 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:333]
  #26 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/sql/manage_storage.php:249]
[2017-04-23 07:10:06] EXCEPTION: (PhutilAggregateException) All storage engines failed to write file:
    - PhabricatorMySQLFileStorageEngine: AphrontSchemaQueryException: #1146: Table 'phabricator_auth.auth_hmackey' doesn't exist
    - PhabricatorLocalDiskFileStorageEngine: AphrontSchemaQueryException: #1146: Table 'phabricator_auth.auth_hmackey' doesn't exist at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:375]
arcanist(head=master, ref.master=146693307f60), phabricator(head=master, ref.master=bc9291b32708), phutil(head=master, ref.master=6fe33623cda6)
  #0 <#2> AphrontBaseMySQLDatabaseConnection::throwCommonException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:348]
  #1 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryCodeException(integer, string) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:289]
  #2 <#2> AphrontBaseMySQLDatabaseConnection::throwQueryException(mysqli) called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:185]
  #3 <#2> AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [<phutil>/src/xsprintf/queryfx.php:8]
  #4 <#2> queryfx(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:13]
  #5 <#2> queryfx_all(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phutil>/src/xsprintf/queryfx.php:19]
  #6 <#2> queryfx_one(AphrontMySQLiDatabaseConnection, string, string, string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:220]
  #7 <#2> PhabricatorHash::readHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:177]
  #8 <#2> PhabricatorHash::getNamedHMACKey(string) called at [<phabricator>/src/infrastructure/util/PhabricatorHash.php:146]
  #9 <#2> PhabricatorHash::digestWithNamedKey(string, string) called at [<phabricator>/src/applications/files/engine/PhabricatorFileStorageEngine.php:374]
  #10 <#2> PhabricatorFileStorageEngine::newIntegrityHash(string, PhabricatorFileRawStorageFormat) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:520]
  #11 <#2> PhabricatorFile::writeToEngine(PhabricatorMySQLFileStorageEngine, string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:351]
  #12 PhabricatorFile::buildFromFileData(string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:419]
  #13 PhabricatorFile::newFromFileData(string, array) called at [<phabricator>/src/applications/files/transform/PhabricatorFileImageTransform.php:151]
  #14 PhabricatorFileImageTransform::newFileFromData(string) called at [<phabricator>/src/applications/files/transform/PhabricatorFileThumbnailTransform.php:228]
  #15 PhabricatorFileThumbnailTransform::getDefaultTransform(PhabricatorFile) called at [<phabricator>/src/applications/files/transform/PhabricatorFileTransform.php:27]
  #16 PhabricatorFileTransform::executeTransform(PhabricatorFile) called at [<phabricator>/resources/sql/autopatches/20161005.conpherence.image.2.php:25]
  #17 require_once(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:285]
  #18 PhabricatorStorageManagementAPI::applyPatchPHP(string) called at [<phabricator>/src/infrastructure/storage/management/PhabricatorStorageManagementAPI.php:241]
  #19 PhabricatorStorageManagementAPI::applyPatch(PhabricatorStoragePatch) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:1093]
  #20 PhabricatorStorageManagementWorkflow::doUpgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:840]
  #21 PhabricatorStorageManagementWorkflow::upgradeSchemata(array, NULL, boolean, boolean) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementUpgradeWorkflow.php:78]
  #22 PhabricatorStorageManagementUpgradeWorkflow::didExecute(PhutilArgumentParser) called at [<phabricator>/src/infrastructure/storage/management/workflow/PhabricatorStorageManagementWorkflow.php:107]
  #23 PhabricatorStorageManagementWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:441]
  #24 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:333]
  #25 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/sql/manage_storage.php:249]

}

Pabricator tells me I am missing these patches:
Run the storage upgrade script to upgrade databases (host "localhost" is out of date). Missing patches: phabricator:20161005.conpherence.image.2.php, phabricator:20161011.conpherence.ngrams.php, phabricator:20161011.conpherence.ngrams.sql, phabricator:20161012.cal.01.import.sql, phabricator:20161012.cal.02.importxaction.sql, phabricator:20161012.cal.03.eventimport.sql, phabricator:20161013.cal.01.importlog.sql, phabricator:20161016.conpherence.imagephids.sql, phabricator:20161025.phortune.contact.1.sql, phabricator:20161025.phortune.merchant.image.1.sql, phabricator:20161026.calendar.01.importtriggers.sql, phabricator:20161027.calendar.01.externalinvitee.sql, phabricator:20161029.phortune.invoice.1.sql, phabricator:20161031.calendar.01.seriesparent.sql, phabricator:20161031.calendar.02.notifylog.sql, phabricator:20161101.calendar.01.noholiday.sql, phabricator:20161101.calendar.02.removecolumns.sql, phabricator:20161104.calendar.01.availability.sql, phabricator:20161104.calendar.02.availdefault.sql, phabricator:20161115.phamepost.01.subtitle.sql, phabricator:20161115.phamepost.02.header.sql, phabricator:20161121.cluster.01.hoststate.sql, phabricator:20161124.search.01.stopwords.sql, phabricator:20161125.search.01.stemmed.sql, phabricator:20161130.search.01.manual.sql, phabricator:20161130.search.02.rebuild.php, phabricator:20161210.dashboards.01.author.sql, phabricator:20161210.dashboards.02.author.php, phabricator:20161211.menu.01.itemkey.sql, phabricator:20161211.menu.02.itemprops.sql, phabricator:20161211.menu.03.order.sql, phabricator:20161212.dashboardpanel.01.author.sql, phabricator:20161212.dashboardpanel.02.author.php, phabricator:20161212.dashboards.01.icon.sql, phabricator:20161213.diff.01.hunks.php, phabricator:20161216.dashboard.ngram.01.sql, phabricator:20161216.dashboard.ngram.02.php, phabricator:20170106.menu.01.customphd.sql, phabricator:20170109.diff.01.commit.sql, phabricator:20170119.menuitem.motivator.01.php, phabricator:20170131.dashboard.personal.01.php, phabricator:20170301.subtype.01.col.sql, phabricator:20170301.subtype.02.default.sql, phabricator:20170301.subtype.03.taskcol.sql, phabricator:20170301.subtype.04.taskdefault.sql, phabricator:20170303.people.01.avatar.sql, phabricator:20170313.reviewers.01.sql, phabricator:20170316.rawfiles.01.php, phabricator:20170320.reviewers.01.lastaction.sql, phabricator:20170320.reviewers.02.lastcomment.sql, phabricator:20170320.reviewers.03.migrate.php, phabricator:20170322.reviewers.04.actor.sql, phabricator:20170328.reviewers.01.void.sql, phabricator:20170406.hmac.01.keystore.sql, phabricator:20170410.calendar.01.repair.php, phabricator:20170412.conpherence.01.picturecrop.sql, phabricator:20170413.conpherence.01.recentparty.sql, phabricator:20170417.files.ngrams.sql, phabricator:20170418.1.application.01.xaction.sql, phabricator:20170418.1.application.02.edge.sql, phabricator:20170418.files.isDeleted.sql, phabricator:20170419.app.01.table.sql, phabricator:20170419.thread.01.behind.sql, phabricator:20170419.thread.02.status.sql, phabricator:20170419.thread.03.touched.sql.

Any suggestions on how to fix this?

Event Timeline

This is a bug in the upstream. D17770 (or some similar patch) should be available later today.

As currently written, D17770 just skips this migration. If you skip this migration, existing rooms in Conpherence with low-resolution profile images will lose their images during the upgrade, and you'll need to update them with new images (the migration corresponds to us making profile image thumbnails much higher-resolution, and also changing how the images are stored).

I think this isn't too big of an issue (and that doing this migration more faithfully is so much more work that it probably isn't worthwhile), but let us know if this is a concern.

Before that lands, you can work around this issue by making a similar change to resources/sql/autopatches/20161005.conpherence.image.2.php -- just delete the whole body of the file -- then running bin/storage upgrade again.

epriestley renamed this task from Error running Storage Upgrade to Old migration resizes and writes new Conpherence room images, but Files have changed since then.Apr 23 2017, 11:53 AM

Evan, rP52c4715bbcfb solvedthe issue. Losing low-resolution profile images is no problem.
Thanks for the quick solution!