Page MenuHomePhabricator

Parsing commits that do not contain available users breaks the task
Closed, InvalidPublic

Description

We have started using phabricator and noticed that some tasks linger in phabricator/daemon/ and dont get finished.

ID	Class	                                   Owner	                           Expires	        Failures	
807459	PhabricatorRepositoryCommitHeraldWorker	13545:1416311621:ubuntu:2	-101556	3000	143	View Task
807465	PhabricatorRepositoryCommitHeraldWorker	13598:1416311634:ubuntu:2	-101543	3000	143	View Task
807524	PhabricatorRepositoryCommitHeraldWorker	13616:1416311638:ubuntu:2	-101539	3000	143	View Task
807531	PhabricatorRepositoryCommitHeraldWorker	13618:1416311638:ubuntu:2	-101539	3000	143	View Task

This seems to only happen to tasks from users that dont exist in Phabricator. After some debugging and Irc chatting. I've found that when:

./scripts/repository/reparse.php rEB96bb4893bd14998cf426787617c91c6ca641be97 --herald

[                                                                  ]   0.0%[2014-11-19 16:58:26] EXCEPTION: (AphrontParameterQueryException) Array for %Ls conversion is empty. Query: SELECT * FROM %s WHERE userPHID in (%Ls) %Q at [<phutil>/src/xsprintf/qsprintf.php:252]
  #0 _qsprintf_check_type(array, string, string) called at [<phutil>/src/xsprintf/qsprintf.php:129]
  #1 xsprintf_query(AphrontMySQLiDatabaseConnection, string, integer, array, integer) called at [<phutil>/src/xsprintf/xsprintf.php:63]
  #2 xsprintf(string, AphrontMySQLiDatabaseConnection, array) called at [<phutil>/src/xsprintf/qsprintf.php:64]
  #3 qsprintf(AphrontMySQLiDatabaseConnection, string, string, array, string)
  #4 call_user_func_array(string, array) called at [<phutil>/src/xsprintf/queryfx.php:5]
  #5 queryfx(AphrontMySQLiDatabaseConnection, string, string, array, string)
  #6 call_user_func_array(string, array) called at [<phutil>/src/xsprintf/queryfx.php:16]
  #7 queryfx_all(AphrontMySQLiDatabaseConnection, string, string, array, string)
  #8 call_user_func_array(string, array) called at [<phutil>/src/aphront/storage/connection/AphrontDatabaseConnection.php:20]
  #9 AphrontDatabaseConnection::queryData(string, string, array, string)
  #10 call_user_func_array(array, array) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:509]
  #11 LiskDAO::loadRawDataWhere(string, array)
  #12 call_user_func_array(array, array) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:452]
  #13 LiskDAO::loadAllWhere(string, array) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:196]
  #14 PhabricatorFeedStoryPublisher::filterSubscribedPHIDs(array) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:124]
  #15 PhabricatorFeedStoryPublisher::publish() called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2293]
  #16 PhabricatorApplicationTransactionEditor::publishFeedStory(PhabricatorRepositoryCommit, array, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:807]
  #17 PhabricatorApplicationTransactionEditor::applyTransactions(PhabricatorRepositoryCommit, array) called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitHeraldWorker.php:69]
  #18 PhabricatorRepositoryCommitHeraldWorker::parseCommit(PhabricatorRepository, PhabricatorRepositoryCommit) called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitParserWorker.php:44]
  #19 PhabricatorRepositoryCommitParserWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:87]
  #20 PhabricatorWorker::executeTask() called at [<phabricator>/scripts/repository/reparse.php:281]

Event Timeline

elastinuno raised the priority of this task from to Needs Triage.
elastinuno updated the task description. (Show Details)
elastinuno added a subscriber: elastinuno.
epriestley added a subscriber: epriestley.

Support Impact I haven't triaged, but this looks like a bug with no recourse.

I do not know how to reproduce this issue as reported. For example, I import the Phabricator production instance into my local dev install. I do have a btrahan account there, but otherwise there is no user account overlap. Commits seem to import fine:

Screen_Shot_2014-12-22_at_11.56.43_AM.png (1×1 px, 601 KB)

Does anyone have steps on how to get this issue to reproduce?

The issue is not to import the commits into phabricator. But when the workers start to parse the commits for "whatever". If you go to the deamons task list is everything fine ?

Try to run ./scripts/repository/reparse.php rEB96bb4893bd14998cf426787617c91c6ca641be97 --herald for a commit that there is no user.

What we did on our case was to import a repository before we created the users. Doing that spawned lots of tasks that would not be finished.

Yes, everything is fine. Just an FYI that importing the commits into Phabricator is done via the various worker parsing...

What Herald rule(s) do you have that run against commits?

https://secure.phabricator.com/book/phabcontrib/article/bug_reports/#unreproducible-problems

We had no herald rules it was a clean start.

We just noticed that our "recent activity" was polluted with old commits. From there a quick look at the deamons and they where stuck.

After that it I tried with the help of IRC channel to find the error. When I used the reparsing tool for those commits the script started to give that error. We figured by then that It must have been the users so we started by adding the users and their emails. The problem disappeared after we did this and then reparsed all the commits with the script.

Yes, everything is fine. Just an FYI that importing the commits into Phabricator is done via the various worker parsing...

Yes this is true. But there are several steps to import a commit and you dont have to make all of them for the commit to appear. So only part of the parsing was being successful.

Thanks.

btrahan claimed this task.

I went so far as to re-install Phabricator from scratch and was able to successfully fully import the Phabricator repository again.

Please do re-open with explicit reproduction instructions that work and I am happy to fix this. Thanks!