Have `bin/phd stop` look for daemons and warn you about them, or nuke them with some flag
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	epriestley
	Jan 22 2013, 11:41 AM

Description

Sometimes, for whatever reason, daemon processes survive phd stop. After killing daemons we know about, we should run ps auxwww or similar and look for still-living daemons, and warn the user:

WARNING: Some daemons seem to have survived:

  2932  php ./launch_daemon.php PhabricatorGarbageCollectorDaemon --daemonize
        --load-phutil-library=/core/arcanist/src
        --load-phutil-library=/core/phabricator/src
        --conduit-uri=https://secure.phabricator.com/api/
        --log=/var/tmp/phd/log/daemons.log --phd=/var/tmp/phd/pid

Use `phd stop --force` to forcibly kill these processes.

Then, with --force, we should kill them.

Revisions and Commits

rPHU libphutil
	D10357	rPHUae43ce54f61f Daemons - add phd-daemon support for findRunningDaemons
	Restricted Differential Revision	rPHUc5f85ae1056c Add a method to identify running daemons in the process table
rP Phabricator
	D10386	rP7ecbc707846f Daemons - handle daemons that can't be killed a bit better
	D10368	rP2fdd7f0f3d2e Daemons - better handle rogue daemons from phd
	D10354	rPd13d6963dde0 phd - make stop / restart savvy to daemons without pid files

Event Timeline

epriestley claimed this task.Jan 22 2013, 11:41 AM

epriestley triaged this task as Normal priority.

epriestley added a project: Daemons.

epriestley added a subscriber: epriestley.

epriestley added a project: OS Students 2013.Feb 18 2013, 2:39 PM

epriestley removed epriestley as the assignee of this task.Feb 19 2013, 3:50 PM

nickpell claimed this task.Mar 7 2013, 3:52 PM

To reproduce this easily, do:

phabricator/ $ bin/phd launch nice # launches a "nice" daemon that does nothing

Normally, phd stop should be able to stop this daemon.

To reproduce the problem, launch a daemon and then remove the PID file:

rm -rf /var/tmp/phd/pid/* # or wherever these live on your system

Now, phd stop should fail to find the daemon.

You can launch several "nice" daemons with:

phabricator/ $ ./bin/phd launch 4 nice

What is the correct way to distinguish between Phabricator daemons and other processes in the output from ps auxwww? It seems like you would need a filter that wouldn't delete other processes by mistake.

I think looking for the strings exec_daemon.php or launch_daemon.php is good for now. If you want to be extra paranoid, you could additionally require that --daemonize (keeps it from killing phd debug) and --load-phutil-library= (keeps it from killing scripts which happen to be named exec_daemon.php) be present.

The kill code I mentioned is in PhutilDaemonOverseer->annihilateProcessGroup() (in libphutil/).

epriestley claimed this task.Apr 11 2013, 4:20 PM

epriestley added a subscriber: nickpell.

epriestley edited this Maniphest Task.Apr 11 2013, 4:46 PM

epriestley edited this Maniphest Task.Apr 11 2013, 5:02 PM

joshuaspence added a subscriber: joshuaspence.Apr 7 2014, 8:58 AM

D5663 added a reasonable-looking way to detect these processes, but bin/phd never got the glue code to call it.
This task is otherwise still relevant:
- Add code in bin/phd stop and bin/phd restart to warn the user that there are processes which look like daemons that don't have corresponding PID files, and that they can use --force to terminate them.
- With --force, terminate these processes.
- The steps in https://secure.phabricator.com/T2374#8 should still let you reproduce this in modern phd.