Some of the Phacility cluster UX (allocating instances, triggering backups) would benefit from having a workflow like this available.

I noticed this as well, whilst testing in a small VM. I'd recommend at least 2GB.

We have flood, free, cancel and retry workflows now; the exec (run a task by ID in the foreground) and kill (bulk-cancel tasks by query-like selection) workflows described in the original description would probably be the most useful additions at this point. kill might not be a very good name, the desired behavior is more like cancel-all-tasks-matching [conditions].

Thanks !!!for this - https://github.com/phacility/phabricator/blob/master/resources/celerity/map.php.

When I ignored these two issues, the issue nr 1 - "Daemons are running as the wrong user" got resolved.

Thanks for your patience.

This is a configuration issue that we can't reproduce.

I have configured it to use SSL over port 465. One mail got delivered yesterday using this configuration.

You have at least one error in your configuration:

Thanks for looking into this. I think I've configured SMTP correctly. Also I did asked a question http://stackoverflow.com/questions/27742259/external-smtp-gmail-configured-on-phabricator-not-working/27772253?iemail=1&noredirect=1#27772253

Daemon 370 STDE [Wed, 07 Jan 2015 22:36:31 +0530] [2015-01-07 22:36:31] EXCEPTION: (PhutilProxyException) Error while executing task ID 54521 from queue. {>} (phpmailerException) SMTP Error: Could not authenticate. at [<phabricator>/externals/phpmailer/class.phpmailer.php:798]

sure, here is the o/p of phabricator/ $ ./bin/phd log

I'm going to kick this out of Phacility, we've got everything we need now.

One thing I havent quite worked out yet is whether the daemon user (phd in my case) requires a shell and/or home directory. If possible, it's nice to be able to set the shell to something like /bin/false.

We could add a --force mode which would kill the daemons regardless of the status of MySQL. That seems like a reasonable compromise.

The site doesn't work at all with mysql down (and it doesn't look pretty
either)

I think it should loudly complain MySQL is down. Optionally / maybe not a good idea, it can include instructions on how to kill the daemons otherwise and then clean up the MySQL entries that will (likely) be there when MySQL restarts.

Just ran into this

Closed by commit rP2fe553761ae7.

Possibly we should just prevent the daemons from launching if this isn't defined, although maybe that's too severe. We could look at that if more users run into trouble.

Seems to see fixed.

Sounds good, thanks!

I think the two diffs mostly get us there... Still outstanding would be:

No object PHID is 100% fine for now. In the very long run it would be nice to get the "real" object PHID (e.g., the original task or whatever that the user commented on), but for now I just need the slot to put Phacility instance PHIDs in so I can show them on the console. It's fine if core/upstream support is spotty/limited/imaginary.

Some early notes and questions looking at this -

Closed by commit rP2fedb6f941d8.

I should add that the current setup check only alerts if there's a mismatch between config and running daemon.
If phd.user is not configured at all (default) it will not trigger.

Oh sorry my bad. The context slipped by me.

There were two separate patches to the daemons -- I think @joshuaspence was talking about D10766 (affecting the execution order of tasks), not D10895 (affecting performance for large queues).

I filed T6768 to eventually produce a cleaner fix for this issue; D10998 should work, it's just not very elegant.

Confirmed this is a duplicate of D10998

Perhaps I'm running into T6456?

We also are running the patch and it has been great, could this in some way be related to carry over daemon work from before getting in a weird state? We did carry over a large backlog though during upgrade (500k+) and PhD caught up like a champ so I dont know.

Perhaps there should be a minimum memory requirement documented here?

it is a small vm for testing (only phabricator is running on it)

Can you tell us more about your environment? How much memory does this machine have? What is the OS?

Thanks for the quick turnaround on this!

While not required, this would be materially helpful on the instance management console, so queued operations could be made visible.

This should be resolved in HEAD. Thanks for the report!

Closed by commit rP9a7383121dbb.

I'm just going to move this to the CLI. See also T3554.

Let us know if you're still seeing issues with large queues after those patches, but I think the meat of this issue is fixed.

I can't reproduce this, but we'll keep an eye on it.

Well, it should have worked there...

I'm running into this problem too, with Mercurial 3.1.2 (with Python 2.7.6) and phabricator 3463ce8a514f87287cd961ded284e60153e851d8 (from Fri 3rd Oct).

hg heads -c --template "{branch}\n" | wc -l

Support Impact This is confusing.

Support Impact This is wildly difficult to diagnose because it manifests as a mysterious hang.