not able to delete tasks from phd queue
Closed, InvalidPublic
Actions

Assigned To

Authored By

	lfarkas
	Oct 16 2015, 12:24 PM

Description

itry different ways to add repositories to phabricator. after a few try i delete most of the repositories. but unfortunately when i destroy the repositories the task belong to these repositories still remain in the queue and never be able to finish so never deleted from the queue. is there any way to delete them. tasks like:

PhabricatorRepositoryGitCommitMessageParserWorker
PhabricatorRepositoryCommitHeraldWorker
PhabricatorRepositoryGitCommitChangeParserWorker
PhabricatorRepositoryCommitOwnersWorker

or should i've to delete directly from the mysql db?

Event Timeline

lfarkas created this task.Oct 16 2015, 12:24 PM

lfarkas updated the task description. (Show Details)

lfarkas added a subscriber: lfarkas.

chad added a project: Daemons.Oct 16 2015, 8:08 PM

Are you certain these tasks aren't being removed automatically? The expectation is that they will naturally fail over time when they are unable to load the corresponding commits.

yes i'n certain!
it was running all night and at the morning it's still running. when i look into the daemon's queue i see the reason is failed for each job so will repeat later in an exponential delay time.

Okay, here's what I did to try to reproduce this:

I stopped the daemons.
I created a new repository, with callsign ABCD.
I set it to import a repository from GitHub (I used https://github.com/phacility/libphutil).
I ran bin/repository update ABCD to run discovery on this repository.

$ ./bin/repository update ABCD
Updated repository rABCD.

I verified that the queue populated with a large number of tasks by examining daemon/ in the web UI:

I ran bin/phd debug task to work through this queue in the foreground in debug mode, so I could observe what happened. I saw a few tasks complete successfully, and I ^C'd it after a little while. This left the daemon queue in this state:

I used bin/remove destroy rABCD to destroy the repository. I confirmed the enormous, explicit prompt and warning:

$ ./bin/remove destroy rABCD


                                  uuuuuuu
                               uu###########uu
                            uu#################uu
                           u#####################u
                          u#######################u
                         u#########################u
                         u#########################u
                         u######"   "###"   "######u
                         "####"      u#u       ####"
                          ###u       u#u       u###
                          ###u      u###u      u###
                           "####uu###   ###uu####"
                            "#######"   "#######"
                              u#######u#######u
                               u#"#"#"#"#"#"#u
                    uuu        ##u# # # # #u##       uuu
                   u####        #####u#u#u###       u####
                    #####uu      "#########"     uu######
                  u###########uu    """""    uuuu##########
                  ####"""##########uuu   uu#########"""###"
                   """      ""###########uu ""#"""
                             uuuu ""##########uuu
                    u###uuu#########uu ""###########uuu###
                    ##########""""           ""###########"
                     "#####"                      ""####""
                       ###"                         ####"


 IMPORTANT  DATA WILL BE PERMANENTLY DESTROYED

Objects will be permanently destroyed. There is no way to undo this operation
or ever retrieve this data unless you maintain external backups.

 IMPORTANT  DELETING OBJECTS OFTEN BREAKS THINGS

Destroying objects may cause related objects to stop working, and may leave
scattered references to objects which no longer exist. In most cases, it is
much better to disable or archive objects instead of destroying them. This
risk is greatest when deleting complex or highly connected objects like
repositories, projects and users.

These tattered edges are an expected consquence of destroying objects, and
the Phabricator upstream will not help you fix them. We strongly recomend
disabling or archiving objects instead.

This object will be destroyed forever:

    - rABCD (PhabricatorRepository) rABCD Abcd


    Are you absolutely certain you want to destroy this object? [y/N] y

Destroying objects...
Destroying PhabricatorRepository rABCD...
Permanently destroyed 1 object(s).

I ran bin/phd debug task again to observe the queue behavior. This time, I saw permanent failures:

[2015-10-19 09:36:09] EXCEPTION: (PhutilProxyException) Permanent failure while executing Task ID 474622. {>} (PhabricatorWorkerPermanentFailureException) Commit "84537" does not exist. at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitParserWorker.php:23]
arcanist(head=master, ref.master=6c7def560d5e), phabricator(head=master, ref.master=057d62d570d4, custom=24), phutil(head=master, ref.master=db3667a62f66)
  #0 <#2> PhabricatorRepositoryCommitParserWorker::loadCommit() called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitParserWorker.php:31]
  #1 <#2> PhabricatorRepositoryCommitParserWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:122]
  #2 <#2> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:171]
  #3 <#2> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
  #4 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:183]
  #5 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:125]

I observed a corresponding decrease in the number of tasks in queue. Note that there are 10 fewer GitCommitMessage tasks in this screenshot:

I started the daemons again (with bin/phd start) and let them run for a while. I continued to observe the queue steadily empty. Here's a screenshot after about a minute (note that there are ~100 fewer tasks in the queue):

I observed the queue continuing to flush on its own. Here's another screenshot a little later:

So I can't reproduce the behavior you describe. Instead, the behavior I observe appears to be the expected behavior: these tasks naturally fail over time when they are unable to load the corresponding commits.

Can you show me what I can do differently to reproduce the problem you're encountering?

may be i was to lazy to wait enough. it was a repo with 40.000+ changeset. and one more thing i add the repo with V callsign then delete it and add again with the same callsign. but may be it just dump the exception in the log many thousand time and i've to wait more.

Okay. This report doesn't appear to describe a reproducible problem. See:

https://secure.phabricator.com/book/phabcontrib/article/bug_reports/

	F896715: Screen Shot 2015-10-19 at 9.32.38 AM.png
	Oct 19 2015, 4:42 PM

	F896724: Screen Shot 2015-10-19 at 9.36.48 AM.png
	Oct 19 2015, 4:42 PM

	F896728: Screen Shot 2015-10-19 at 9.40.49 AM.png
	Oct 19 2015, 4:42 PM

	F896717: Screen Shot 2015-10-19 at 9.34.11 AM.png
	Oct 19 2015, 4:42 PM

	F896726: Screen Shot 2015-10-19 at 9.38.20 AM.png
	Oct 19 2015, 4:42 PM

not able to delete tasks from phd queueClosed, InvalidPublicActions

Description

Event Timeline

not able to delete tasks from phd queue
Closed, InvalidPublic
Actions