Page MenuHomePhabricator

missing mysql extension incredibly difficult to debug
Closed, DuplicatePublic

Description

I'm migrating our Phabricator install from 14.04 to 16.04 which includes a move to systemd and other stuff, and it's ansible deployed.

While the web app, aphlict and sshd run out of the box as-is, phd would not come up for nothing.

Trying to run phd manually blurts out the following:

root@granite:/opt/phabricator/phabricator# bin/phd
[2017-07-28 08:21:06] EXCEPTION: (PhabricatorClusterStrandedException) Unable to establish a connection to any database host (while trying "phabricator_config"). All masters and replicas are completely unreachable. at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:140]
arcanist(head=stable, ref.master=7bb8dbabce83, ref.stable=9a9009f34d46), phabricator(head=stable, ref.master=0d8f4170f4dc, ref.stable=81b5f90dd241), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
  #0 PhabricatorLiskDAO::raiseUnreachable(string) called at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:111]
  #1 PhabricatorLiskDAO::newClusterConnection(string, string, string) called at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:66]
  #2 PhabricatorLiskDAO::establishLiveConnection(string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1008]
  #3 LiskDAO::establishConnection(string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:516]
  #4 LiskDAO::loadRawDataWhere(string, string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:476]
  #5 LiskDAO::loadAllWhere(string, string) called at [<phabricator>/src/infrastructure/env/PhabricatorConfigDatabaseSource.php:19]
  #6 PhabricatorConfigDatabaseSource::loadConfig(string) called at [<phabricator>/src/infrastructure/env/PhabricatorConfigDatabaseSource.php:7]
  #7 PhabricatorConfigDatabaseSource::__construct(string) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:249]
  #8 PhabricatorEnv::buildConfigurationSourceStack(boolean) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:95]
  #9 PhabricatorEnv::initializeCommonEnvironment(boolean) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:75]
  #10 PhabricatorEnv::initializeScriptEnvironment(boolean) called at [<phabricator>/scripts/init/lib.php:22]
  #11 init_phabricator_script(array) called at [<phabricator>/scripts/init/init-script.php:9]
  #12 require_once(string) called at [<phabricator>/scripts/__init_script__.php:3]
  #13 require_once(string) called at [<phabricator>/scripts/daemon/manage_daemons.php:5]
root@granite:/opt/phabricator/phabricator#

Meanwhile, the web app has no trouble connecting to the database.

root@granite:/opt/phabricator/phabricator# bin/config get mysql.host
{
  "config": [
    {
      "key": "mysql.host",
      "source": "local",
      "value": "REDACTED.us-east-1.rds.amazonaws.com",
      "status": "set",
      "errorInfo": null
    },
    {
      "key": "mysql.host",
      "source": "database",
      "value": null,
      "status": "error",
      "errorInfo": "Database source is not configured properly"
    }
  ]
}
root@granite:/opt/phabricator/phabricator#

The "database source is not configured properly" message was my first clue that something was off in the setup, and not with the database.

I was out of ideas so I started kicking every tyre, hoping something would fall out... and indeed it did:

root@granite:/opt/phabricator/phabricator# ./bin/storage upgrade
[2017-07-28 04:24:10] EXCEPTION: (Exception) About to call mysql_connect(), but the PHP MySQL extension is not available! at [<phutil>/src/aphront/storage/connection/mysql/AphrontMySQLDatabaseConnection.php:33]
arcanist(head=stable, ref.master=7bb8dbabce83, ref.stable=9a9009f34d46), phabricator(head=stable, ref.master=0d8f4170f4dc, ref.stable=81b5f90dd241), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
  #0 AphrontMySQLDatabaseConnection::connect() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:101]
  #1 AphrontBaseMySQLDatabaseConnection::establishConnection() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:124]
  #2 AphrontBaseMySQLDatabaseConnection::requireConnection() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:160]
  #3 AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [<phutil>/src/xsprintf/queryfx.php:8]
  #4 queryfx(AphrontMySQLDatabaseConnection, string) called at [<phabricator>/scripts/sql/manage_storage.php:162]
root@granite:/opt/phabricator/phabricator#

Hold on. the mysql extension is very much installed, otherwise the web app would not run.

Oh wait a minute...

root@granite:/opt/phabricator/phabricator# php -v
PHP 7.1.7-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Jul  7 2017 09:41:45) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.7-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
root@granite:/opt/phabricator/phabricator#

There we go. The PHP CLI is the wrong version.

root@granite:/opt/phabricator/phabricator# update-alternatives --set php /usr/bin/php5.6
update-alternatives: using /usr/bin/php5.6 to provide /usr/bin/php (php) in manual mode
root@granite:/opt/phabricator/phabricator# ./bin/storage upgrade
Before running storage upgrades, you should take down the Phabricator web
interface and stop any running Phabricator daemons (you can disable this
warning with --force).

    Are you ready to continue? [y/N] ^C
root@granite:/opt/phabricator/phabricator# bin/config get mysql.host
{
  "config": [
    {
      "key": "mysql.host",
      "source": "local",
      "value": "REDACTED.us-east-1.rds.amazonaws.com",
      "status": "set",
      "errorInfo": null
    },
    {
      "key": "mysql.host",
      "source": "database",
      "value": null,
      "status": "unset",
      "errorInfo": null
    }
  ]
}
root@granite:/opt/phabricator/phabricator#

And phd works as well.

My actual complaint here is the phd error, and the config error very obviously mask the root cause which is cannot call mysql_connect.

Event Timeline

allixsenos renamed this task from wrong php cli version incredibly difficult to debug to missing mysql extension incredibly difficult to debug.Jul 28 2017, 8:30 AM
allixsenos created this task.