Page MenuHomePhabricator

Daemons try and fail to publish transactions for newly created diffs
Closed, ResolvedPublic

Description

You can reproduce this pretty easily, just create a new revision using arc diff and you should find a corresponding error logged on a taskmaster:

[2016-01-25 00:02:22] EXCEPTION: (PhutilProxyException) Permanent failure while executing Task ID 320. {>} (PhabricatorWorkerPermanentFailureException) Unable to load transactions: PHID-XACT-DREV-nhexnck64mngrhz. at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:112]
arcanist(head=master, ref.master=b87138356a9c), phabricator(head=master, ref.master=a9e2e6c5aab9), phutil(head=master, ref.master=f43291e99d36)
  #0 <#2> PhabricatorApplicationTransactionPublishWorker::loadTransactions(DifferentialDiff) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:19]
  #1 <#2> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:122]
  #2 <#2> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:171]
  #3 <#2> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
  #4 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:183]
  #5 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:125]

Underlying issue here is that the transaction is saved as a DifferentialTransaction rather than a DifferentialDiffTransaction.

mysql> select * from differential_transaction where phid = 'PHID-XACT-DREV-nhexnck64mngrhz'\G
*************************** 1. row ***************************
             id: 15
           phid: PHID-XACT-DREV-nhexnck64mngrhz
     authorPHID: PHID-USER-rihx4366f3aczsvc2wtb
     objectPHID: PHID-DIFF-fgt7avmnvsxgcbgzx5ci
     viewPolicy: public
     editPolicy: PHID-USER-rihx4366f3aczsvc2wtb
    commentPHID: NULL
 commentVersion: 0
transactionType: differential:diff:create
       oldValue: null
       newValue: true
  contentSource: {"source":"conduit","params":[]}
       metadata: []
    dateCreated: 1453698141
   dateModified: 1453698141
1 row in set (0.00 sec)

Event Timeline

epriestley claimed this task.
epriestley added a subscriber: epriestley.

This should be resolved by D15112. I've made a note in T9789 to catch this class of issue sooner before we make the consequences more dire by pursuing changes planned there.

I'm seeing these errors failing regularly:

[27-Jul-2017 00:14:41 Etc/UTC] [2017-07-27 00:14:41] EXCEPTION: (PhutilProxyException) Task "31509932" encountered a permanent failure and was cancelled. {>} (PhabricatorWorkerPermanentFailureException) Unable to load transactions: PHID-XACT-TASK-u3qhmjddt4jzimr. at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:112]
[27-Jul-2017 00:14:41 Etc/UTC] arcanist(head=stable, ref.master=165df12046e5, ref.stable=b9e79d14c143), phabricator(head=stable, ref.master=bddd1da053e8, ref.stable=5ad86a1052d1), phlab(head=master, ref.master=2518802ab84e), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
[27-Jul-2017 00:14:41 Etc/UTC]   #0 <#3> PhabricatorApplicationTransactionPublishWorker::loadTransactions(ManiphestTask) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:19]
[27-Jul-2017 00:14:41 Etc/UTC]   #1 <#3> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:123]
[27-Jul-2017 00:14:41 Etc/UTC]   #2 <#3> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:171]
[27-Jul-2017 00:14:41 Etc/UTC]   #3 <#3> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
[27-Jul-2017 00:14:41 Etc/UTC]   #4 phlog(PhutilProxyException) called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:34]
[27-Jul-2017 00:14:41 Etc/UTC]   #5 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:219]
[27-Jul-2017 00:14:41 Etc/UTC]   #6 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:131]
[27-Jul-2017 00:14:41 Etc/UTC] [2017-07-27 00:14:41] EXCEPTION: (PhutilProxyException) Task "31509932" encountered a permanent failure and was cancelled. {>} (PhabricatorWorkerPermanentFailureException) Unable to load transactions: PHID-XACT-TASK-u3qhmjddt4jzimr. at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:112]
[27-Jul-2017 00:14:41 Etc/UTC] arcanist(head=stable, ref.master=165df12046e5, ref.stable=b9e79d14c143), phabricator(head=stable, ref.master=bddd1da053e8, ref.stable=5ad86a1052d1), phlab(head=master, ref.master=2518802ab84e), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
[27-Jul-2017 00:14:41 Etc/UTC]   #0 <#3> PhabricatorApplicationTransactionPublishWorker::loadTransactions(ManiphestTask) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:19]
[27-Jul-2017 00:14:41 Etc/UTC]   #1 <#3> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:123]
[27-Jul-2017 00:14:41 Etc/UTC]   #2 <#3> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:171]
[27-Jul-2017 00:14:41 Etc/UTC]   #3 <#3> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
[27-Jul-2017 00:14:41 Etc/UTC]   #4 phlog(PhutilProxyException) called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:34]
[27-Jul-2017 00:14:41 Etc/UTC]   #5 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:219]
[27-Jul-2017 00:14:41 Etc/UTC]   #6 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:131]
[31-Jul-2017 00:03:01 Etc/UTC] [2017-07-31 00:03:01] EXCEPTION: (PhutilProxyException) Task "31574777" encountered a permanent failure and was cancelled. {>} (PhabricatorWorkerPermanentFailureException) Unable to load transactions: PHID-XACT-DREV-rxbhjaqqbo4fkzc. at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:112]
[31-Jul-2017 00:03:01 Etc/UTC] arcanist(head=stable, ref.master=165df12046e5, ref.stable=b9e79d14c143), phabricator(head=stable, ref.master=bddd1da053e8, ref.stable=5ad86a1052d1), phlab(head=master, ref.master=2518802ab84e), phutil(head=stable, ref.master=0a4487d37cd7, ref.stable=30065f1c5a39)
[31-Jul-2017 00:03:01 Etc/UTC]   #0 <#3> PhabricatorApplicationTransactionPublishWorker::loadTransactions(DifferentialRevision) called at [<phabricator>/src/applications/transactions/worker/PhabricatorApplicationTransactionPublishWorker.php:19]
[31-Jul-2017 00:03:01 Etc/UTC]   #1 <#3> PhabricatorApplicationTransactionPublishWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:123]
[31-Jul-2017 00:03:01 Etc/UTC]   #2 <#3> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:171]
[31-Jul-2017 00:03:01 Etc/UTC]   #3 <#3> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
[31-Jul-2017 00:03:01 Etc/UTC]   #4 phlog(PhutilProxyException) called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:34]
[31-Jul-2017 00:03:01 Etc/UTC]   #5 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:219]
[31-Jul-2017 00:03:01 Etc/UTC]   #6 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:131]