Currently, the ./bin/repository parents script is failing on a large repository that we have. I'm pretty sure that the reason that this script is failing is because it takes > 4 hours to execute and our DB goes down for backup every 4 hours.
> ./bin/repository parents GAF
Rebuilding "rGAF"...
Rebuilding branch "master"...
Found 83,756 total commit(s); updating...
[2014-06-01 17:20:02] EXCEPTION: (AphrontQueryException) #1317: Query execution was interrupted at [/usr/src/libphutil/src/aphront/storage/connection/mysql/AphrontMySQLDatabaseConnectionBase.php:311]
#0 AphrontMySQLDatabaseConnectionBase::throwQueryCodeException(1317, Query execution was interrupted) called at [/usr/src/libphutil/src/aphront/storage/connection/mysql/AphrontMySQLDatabaseConnectionBase.php:278]
#1 AphrontMySQLDatabaseConnectionBase::throwQueryException(Object mysqli) called at [/usr/src/libphutil/src/aphront/storage/connection/mysql/AphrontMySQLDatabaseConnectionBase.php:184]
#2 AphrontMySQLDatabaseConnectionBase::executeRawQuery(SELECT id, commitIdentifier FROM `repository_commit`
WHERE commitIdentifier IN ('7501a8790ef92f4f690d74eb4132e053580ddcde', 'e89617fe337dd4d76fa14bc7bad2bf4d4c875b08') AND repositoryID = 393) called at [/usr/src/libphutil/src/xsprintf/queryfx.php:9]
#3 queryfx(Object AphrontMySQLiDatabaseConnection, SELECT id, commitIdentifier FROM %T
WHERE commitIdentifier IN (%Ls) AND repositoryID = %d, repository_commit, Array of size 2 starting with: { 0 => 7501a8790ef92f4f690d74eb4132e053580ddcde }, 393)
#4 call_user_func_array(queryfx, Array of size 5 starting with: { 0 => Object AphrontMySQLiDatabaseConnection }) called at [/usr/src/libphutil/src/xsprintf/queryfx.php:25]
#5 queryfx_all(Object AphrontMySQLiDatabaseConnection, SELECT id, commitIdentifier FROM %T
WHERE commitIdentifier IN (%Ls) AND repositoryID = %d, repository_commit, Array of size 2 starting with: { 0 => 7501a8790ef92f4f690d74eb4132e053580ddcde }, 393) called at [/usr/src/phabricator/src/applications/repository/management/PhabricatorRepositoryManagementParentsWorkflow.php:113]
#6 PhabricatorRepositoryManagementParentsWorkflow::rebuildRepository(Object PhabricatorRepository) called at [/usr/src/phabricator/src/applications/repository/management/PhabricatorRepositoryManagementParentsWorkflow.php:43]
#7 PhabricatorRepositoryManagementParentsWorkflow::execute(Object PhutilArgumentParser) called at [/usr/src/libphutil/src/parser/argument/PhutilArgumentParser.php:396]
#8 PhutilArgumentParser::parseWorkflowsFull(Array of size 14 starting with: { PhabricatorRepositoryManagementCacheWorkflow => Object PhabricatorRepositoryManagementCacheWorkflow }) called at [/usr/src/libphutil/src/parser/argument/PhutilArgumentParser.php:292]
#9 PhutilArgumentParser::parseWorkflows(Array of size 14 starting with: { PhabricatorRepositoryManagementCacheWorkflow => Object PhabricatorRepositoryManagementCacheWorkflow }) called at [/usr/src/phabricator/scripts/repository/manage_repositories.php:22]I think that the proper solution here would be to provide a --background option (similar to the existing option for ./bin/search index) such that tasks will be queued for the PhabricatorTaskmasterDaemon instances to complete. There would be a few advantages here:
- Speed: The whole process should complete faster because it will be able to be executed in parallel.
- Fault tolerance: The process will gain fault tolerance which is already built-in to the task queue.