Upgrading phabricator resulted in excess tables/table data
Closed, ResolvedPublic

Description

When upgrading (git pull on libphutil, arcanist, phabricator) bin/storage upgrade resulted in throwing these errors.

Phabricator version info is:

phabricator 1cd64f9975d66a5fff516cd064cd076b0eb7b07f (Fri, Dec 16)
arcanist e17fe43ca3fe6dc6dd0b5ce056f56310ea1d3d51 (Oct 22 2016)
phutil 0ae0cc00acb1413c22bfe3384fd6086ade4cc206 (Sat, Dec 10)

(Last upgrade before that was about 4 weeks ago)

Database           Table                            Name        Issues
phabricator_search
Target                                                          Error
phabricator_search.search_profilepanelconfiguration.key_profile #1072: Key column 'panelOrder' doesn't exist in table

Failed to make some schema adjustments, detailed above.
For help troubleshooting adjustments, see "Managing Storage Adjustments" in the documentation.

Target                                                                 Error
phabricator_dashboard.dashboard.authorPHID                             Surplus
phabricator_dashboard.dashboard.icon                                   Surplus
phabricator_dashboard.dashboard_dashboard_ngrams                       Surplus
phabricator_dashboard.dashboard_dashboardpanel_ngrams                  Surplus
phabricator_dashboard.dashboard_panel.authorPHID                       Surplus
phabricator_search.search_profilepanelconfiguration.menuItemKey        Surplus
phabricator_search.search_profilepanelconfiguration.menuItemOrder      Surplus
phabricator_search.search_profilepanelconfiguration.menuItemProperties Surplus
phabricator_search.search_profilepanelconfiguration.panelKey           Missing
phabricator_search.search_profilepanelconfiguration.panelOrder         Missing
phabricator_search.search_profilepanelconfiguration.panelProperties    Missing
eFrane created this task.Dec 24 2016, 10:41 AM
eFrane updated the task description. (Show Details)Dec 24 2016, 10:51 AM
chad added a subscriber: chad.Dec 24 2016, 4:43 PM

I have no idea how to reproduce this error. I'd guess something along the lines of take a new version of Phabricator and apply an old upgrade script, but that's really just a guess.

I'd re-download Phabricator and attempt the storage upgrade again.

Can you show us the output of these commands?

phabricator/ $ git status
phabricator/ $ cat src/applications/dashboard/storage/PhabricatorDashboard.php
~/phabricator$ git status
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
~/phabricator$ cat src/applications/dashboard/storage/PhabricatorDashboard.php
<?php

/**
 * A collection of dashboard panels with a specific layout.
 */
final class PhabricatorDashboard extends PhabricatorDashboardDAO
  implements
    PhabricatorApplicationTransactionInterface,
    PhabricatorPolicyInterface,
    PhabricatorFlaggableInterface,
    PhabricatorDestructibleInterface,
    PhabricatorProjectInterface {

  protected $name;
  protected $viewPolicy;
  protected $editPolicy;
  protected $status;
  protected $layoutConfig = array();

  const STATUS_ACTIVE = 'active';
  const STATUS_ARCHIVED = 'archived';

  private $panelPHIDs = self::ATTACHABLE;
  private $panels = self::ATTACHABLE;
  private $edgeProjectPHIDs = self::ATTACHABLE;


  public static function initializeNewDashboard(PhabricatorUser $actor) {
    return id(new PhabricatorDashboard())
      ->setName('')
      ->setViewPolicy(PhabricatorPolicies::POLICY_USER)
      ->setEditPolicy($actor->getPHID())
      ->setStatus(self::STATUS_ACTIVE)
      ->attachPanels(array())
      ->attachPanelPHIDs(array());
  }

  public static function getStatusNameMap() {
    return array(
      self::STATUS_ACTIVE => pht('Active'),
      self::STATUS_ARCHIVED => pht('Archived'),
    );
  }

  public static function copyDashboard(
    PhabricatorDashboard $dst,
    PhabricatorDashboard $src) {

    $dst->name = $src->name;
    $dst->layoutConfig = $src->layoutConfig;

    return $dst;
  }

  protected function getConfiguration() {
    return array(
      self::CONFIG_AUX_PHID => true,
      self::CONFIG_SERIALIZATION => array(
        'layoutConfig' => self::SERIALIZATION_JSON,
      ),
      self::CONFIG_COLUMN_SCHEMA => array(
        'name' => 'text255',
        'status' => 'text32',
      ),
    ) + parent::getConfiguration();
  }

  public function generatePHID() {
    return PhabricatorPHID::generateNewPHID(
      PhabricatorDashboardDashboardPHIDType::TYPECONST);
  }

  public function getLayoutConfigObject() {
    return PhabricatorDashboardLayoutConfig::newFromDictionary(
      $this->getLayoutConfig());
  }

  public function setLayoutConfigFromObject(
    PhabricatorDashboardLayoutConfig $object) {
    $this->setLayoutConfig($object->toDictionary());
    return $this;
  }

  public function getProjectPHIDs() {
    return $this->assertAttached($this->edgeProjectPHIDs);
  }

  public function attachProjectPHIDs(array $phids) {
    $this->edgeProjectPHIDs = $phids;
    return $this;
  }

  public function attachPanelPHIDs(array $phids) {
    $this->panelPHIDs = $phids;
    return $this;
  }

  public function getPanelPHIDs() {
    return $this->assertAttached($this->panelPHIDs);
  }

  public function attachPanels(array $panels) {
    assert_instances_of($panels, 'PhabricatorDashboardPanel');
    $this->panels = $panels;
    return $this;
  }

  public function getPanels() {
    return $this->assertAttached($this->panels);
  }

  public function isArchived() {
    return ($this->getStatus() == self::STATUS_ARCHIVED);
  }


/* -(  PhabricatorApplicationTransactionInterface  )------------------------- */


  public function getApplicationTransactionEditor() {
    return new PhabricatorDashboardTransactionEditor();
  }

  public function getApplicationTransactionObject() {
    return $this;
  }

  public function getApplicationTransactionTemplate() {
    return new PhabricatorDashboardTransaction();
  }

  public function willRenderTimeline(
    PhabricatorApplicationTransactionView $timeline,
    AphrontRequest $request) {

    return $timeline;
  }


/* -(  PhabricatorPolicyInterface  )----------------------------------------- */


  public function getCapabilities() {
    return array(
      PhabricatorPolicyCapability::CAN_VIEW,
      PhabricatorPolicyCapability::CAN_EDIT,
    );
  }

  public function getPolicy($capability) {
    switch ($capability) {
      case PhabricatorPolicyCapability::CAN_VIEW:
        return $this->getViewPolicy();
      case PhabricatorPolicyCapability::CAN_EDIT:
        return $this->getEditPolicy();
    }
  }

  public function hasAutomaticCapability($capability, PhabricatorUser $viewer) {
    return false;
  }


/* -(  PhabricatorDestructibleInterface  )----------------------------------- */


  public function destroyObjectPermanently(
    PhabricatorDestructionEngine $engine) {

    $this->openTransaction();
      $installs = id(new PhabricatorDashboardInstall())->loadAllWhere(
        'dashboardPHID = %s',
        $this->getPHID());
      foreach ($installs as $install) {
        $install->delete();
      }

      $this->delete();
    $this->saveTransaction();
  }


}
chad added a comment.Dec 26 2016, 8:32 PM

Offhand, It seems your database is on master, and code is on stable.

epriestley closed this task as Resolved.Dec 28 2016, 4:27 PM
epriestley claimed this task.

Yes, this is consistent with running bin/storage upgrade from master, then switching to stable.

If that is what you did, here's why it got you in trouble and what to do about it:

After running bin/storage upgrade at a particular version, you can only safely upgrade Phabricator to a newer version. That is: Phabricator can not rewind or undo migrations. Because stable is behind master, running bin/storage upgrade on master and then switching to stable is effectively a downgrade, and will leave you with the wrong schema (your schema will be ahead of the code).

To avoid this, change to stable before running bin/storage upgrade if you run stable.

Since master has now promoted to stable, your schema should be correct for HEAD of stable. Upgrading again (making sure not to run bin/storage upgrade from master this time!) should resolve these issues.

If this isn't what you did, we can't reproduce the issue so we can't move forward. Since the information you gave us is consistent with this explanation and we haven't seen other users run into similar issues it seems likely that this is what happened, but it's possible that it isn't. Feel free to file a new report with detailed reproduction steps if you don't think this explains things.