I'm migrating our Phabricator install from 14.04 to 16.04 which includes a move to systemd and other stuff, and it's ansible deployed.
While the web app, aphlict and sshd run out of the box as-is, phd would not come up for nothing.
Trying to run phd manually blurts out the following:
```
root@granite:/opt/phabricator/phabricator# bin/phd
[2017-07-28 08:21:06] EXCEPTION: (PhabricatorClusterStrandedException) Unable to establish a connection to any database host (while trying "phabricator_config"). All masters and replicas are completely unreachable. at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:140]
arcanist(head=stable, ref.master=7bb8dbabce83, ref.stable=9a9009f34d46), phabricator(head=stable, ref.master=0d8f4170f4dc, ref.stable=81b5f90dd241), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
#0 PhabricatorLiskDAO::raiseUnreachable(string) called at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:111]
#1 PhabricatorLiskDAO::newClusterConnection(string, string, string) called at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:66]
#2 PhabricatorLiskDAO::establishLiveConnection(string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1008]
#3 LiskDAO::establishConnection(string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:516]
#4 LiskDAO::loadRawDataWhere(string, string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:476]
#5 LiskDAO::loadAllWhere(string, string) called at [<phabricator>/src/infrastructure/env/PhabricatorConfigDatabaseSource.php:19]
#6 PhabricatorConfigDatabaseSource::loadConfig(string) called at [<phabricator>/src/infrastructure/env/PhabricatorConfigDatabaseSource.php:7]
#7 PhabricatorConfigDatabaseSource::__construct(string) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:249]
#8 PhabricatorEnv::buildConfigurationSourceStack(boolean) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:95]
#9 PhabricatorEnv::initializeCommonEnvironment(boolean) called at [<phabricator>/src/infrastructure/env/PhabricatorEnv.php:75]
#10 PhabricatorEnv::initializeScriptEnvironment(boolean) called at [<phabricator>/scripts/init/lib.php:22]
#11 init_phabricator_script(array) called at [<phabricator>/scripts/init/init-script.php:9]
#12 require_once(string) called at [<phabricator>/scripts/__init_script__.php:3]
#13 require_once(string) called at [<phabricator>/scripts/daemon/manage_daemons.php:5]
root@granite:/opt/phabricator/phabricator#
```
Meanwhile, the web app has no trouble connecting to the database.
```
root@granite:/opt/phabricator/phabricator# bin/config get mysql.host
{
"config": [
{
"key": "mysql.host",
"source": "local",
"value": "REDACTED.us-east-1.rds.amazonaws.com",
"status": "set",
"errorInfo": null
},
{
"key": "mysql.host",
"source": "database",
"value": null,
"status": "error",
"errorInfo": "Database source is not configured properly"
}
]
}
root@granite:/opt/phabricator/phabricator#
```
The "database source is not configured properly" message was my first clue that something was off in the setup, and not with the database.
I was out of ideas so I started kicking every tyre, hoping something would fall out... and indeed it did:
```
root@granite:/opt/phabricator/phabricator# ./bin/storage upgrade
[2017-07-28 04:24:10] EXCEPTION: (Exception) About to call mysql_connect(), but the PHP MySQL extension is not available! at [<phutil>/src/aphront/storage/connection/mysql/AphrontMySQLDatabaseConnection.php:33]
arcanist(head=stable, ref.master=7bb8dbabce83, ref.stable=9a9009f34d46), phabricator(head=stable, ref.master=0d8f4170f4dc, ref.stable=81b5f90dd241), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
#0 AphrontMySQLDatabaseConnection::connect() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:101]
#1 AphrontBaseMySQLDatabaseConnection::establishConnection() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:124]
#2 AphrontBaseMySQLDatabaseConnection::requireConnection() called at [<phutil>/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:160]
#3 AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [<phutil>/src/xsprintf/queryfx.php:8]
#4 queryfx(AphrontMySQLDatabaseConnection, string) called at [<phabricator>/scripts/sql/manage_storage.php:162]
root@granite:/opt/phabricator/phabricator#
```
Hold on. the mysql extension is very much installed, otherwise the web app would not run.
Oh wait a minute...
```
root@granite:/opt/phabricator/phabricator# php -v
PHP 7.1.7-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Jul 7 2017 09:41:45) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.1.7-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
root@granite:/opt/phabricator/phabricator#
```
There we go. The PHP CLI is the wrong version.
```
root@granite:/opt/phabricator/phabricator# update-alternatives --set php /usr/bin/php5.6
update-alternatives: using /usr/bin/php5.6 to provide /usr/bin/php (php) in manual mode
root@granite:/opt/phabricator/phabricator# ./bin/storage upgrade
Before running storage upgrades, you should take down the Phabricator web
interface and stop any running Phabricator daemons (you can disable this
warning with --force).
Are you ready to continue? [y/N] ^C
root@granite:/opt/phabricator/phabricator# bin/config get mysql.host
{
"config": [
{
"key": "mysql.host",
"source": "local",
"value": "phabricator.cluster-ck9ubtmzhqol.us-east-1.rds.amazonaws.com",
"status": "set",
"errorInfo": null
},
{
"key": "mysql.host",
"source": "database",
"value": null,
"status": "unset",
"errorInfo": null
}
]
}
root@granite:/opt/phabricator/phabricator#
```
And phd works as well.
My actual complaint here is the `phd` error, and the `config` error very obviously mask the root cause which is `cannot call mysql_connect`.