Paths

Table of Contentst

Start phd daemons as the correctly configured user and refuse otherwise
Audited2fedb6f941d8
Actions

Description

Start phd daemons as the correctly configured user and refuse otherwise

Summary:
Fixes T5196
If no phd.user is configured the behaviour is unchanged besides printing a warning when run as root (Usually i would add an exit(1) here but that would break existing installs who do that).
If phd.user is set and the current user is root it will run the daemon as: su USER -c "command" (I'm not sure if this works for every platform needed)
Otherwise it will refuse to start if configured and current user mismatch.

Test Plan: Stopped & Started phd daemon with various users and different phd.user settings including root

Reviewers: Blessed Reviewers, epriestley

Reviewed By: Blessed Reviewers, epriestley

Subscribers: vinzent, epriestley

Maniphest Tasks: T5196

Differential Revision: https://secure.phabricator.com/D11036

Details

Auditors

btrahan

Provenance

fabe	Authored on
epriestley	Committed on Dec 23 2014, 4:15 PM
epriestley	Pushed on Dec 23 2014, 4:15 PM

Reviewer

Blessed Reviewers

Differential Revision

D11036: Start phd daemons as the correctly configured user and refuse otherwise

Parents

rP6132d8012b02: show the current size limit when a file upload fails

Branches

Unknown

Tags

Unknown

Tasks

T5196: Enforce that phd is run as the phd.user daemon user

Event Timeline

epriestley committed rP2fedb6f941d8: Start phd daemons as the correctly configured user and refuse otherwise (authored by fabe).Dec 23 2014, 4:15 PM

epriestley added a task: T5196: Enforce that phd is run as the phd.user daemon user.

epriestley added a project: Blessed Reviewers.

Daemons don't really seem to work for me anymore?

12:55:01 ~/Dropbox/code/phalanx (T5402)
~> ./bin/phd start
Freeing active task leases...
Freed 0 task lease(s).
Preparing to launch daemons.
NOTE: Logs will appear in '/var/tmp/phd/log/daemons.log'.

Starting daemons as _phd
Launching daemon "PhabricatorRepositoryPullLocalDaemon".
Starting daemons as _phd
Launching daemon "PhabricatorGarbageCollectorDaemon".
Starting daemons as _phd
Launching daemon "PhabricatorTaskmasterDaemon".
Starting daemons as _phd
Launching daemon "PhabricatorTaskmasterDaemon".
Starting daemons as _phd
Launching daemon "PhabricatorTaskmasterDaemon".
Starting daemons as _phd
Launching daemon "PhabricatorTaskmasterDaemon".
Done.
12:55:06 ~/Dropbox/code/phalanx (T5402)
~> ./bin/phd stop
There are no running Phabricator daemons.

The problem might be with the detection code -- maybe it's not seeing them because they're running as a different user? Are they not present in ps auxwww?

Hrrm, we run ps -a ... so it should still be able to find them.

I don't think its starting them correctly

12:59:03 ~/Dropbox/code/phalanx (T5402)
~> ps auxww | grep _phd
btrahan         16068   0.0  0.0  2432772    664 s000  S+   12:59PM   0:00.00 grep _phd
12:59:08 ~/Dropbox/code/phalanx (T5402)
~> ps auxww | grep Task
btrahan         15766   0.0  0.1  2503016  17224 s001  S+   12:34PM   0:07.47 vim PhabricatorWorkerArchiveTaskQuery.php
btrahan         16075   0.0  0.0  2432772    664 s000  S+   12:59PM   0:00.00 grep Task

My ideas for what could be wrong... 1) From the commit description

"If phd.user is set and the current user is root it will run the daemon as: su USER -c "command" (I'm not sure if this works for every platform needed)"

I'm guessing Mac OS X is such a platform?

Another possibility is there's something wrong with the _phd user on my system I setup. Not sure how to diagnose that.

Does phd debug taskmaster work?

This stuff is working OK for me locally on OSX. Let me check the same codepath you're going down, my config is slightly different...

13:01:28 ~/Dropbox/code/phalanx (T5402)
~> ./bin/phd debug taskmaster
Usage Exception: You are trying to run a daemon as a nonstandard user, and `phd` was not able to `sudo` to the correct user.
Phabricator is configured to run daemons as "_phd", but the current user is "btrahan".
Use `sudo` to run as a different user, pass `--as-current-user` to ignore this warning, or edit `phd.user` to change the configuration.

Try with --as-current-user, just as a sanity check?

Stuff is OK for me locally:

epriestley@orbital ~/dev/phabricator $ sudo -En -- ./bin/phd start
Freeing active task leases...
Freed 1 task lease(s).
Preparing to launch daemons.
NOTE: Logs will appear in '/Users/epriestley/dev/core//log/local/phd/daemons.log'.

Starting daemons as epriestley
Launching daemon "PhabricatorRepositoryPullLocalDaemon".
Starting daemons as epriestley
Launching daemon "PhabricatorGarbageCollectorDaemon".
Starting daemons as epriestley
Launching daemon "PhabricatorTaskmasterDaemon".
Done.
epriestley@orbital ~/dev/phabricator $ ./bin/phd status
ID Host      PID   Started                 Daemon                               Arguments
24 localhost 35171 Dec 23 2014, 1:03:03 PM PhabricatorRepositoryPullLocalDaemon          
25 localhost 35175 Dec 23 2014, 1:03:04 PM PhabricatorGarbageCollectorDaemon             
26 localhost 35184 Dec 23 2014, 1:03:04 PM PhabricatorTaskmasterDaemon

Random thought is that maybe you forgot to bin/storage upgrade first, so the new logging isn't working? Are the daemons showing up in the console / anything in phd log?

(Sorry, went and had lunch)

It works with --as-current-user
Storage is all updated; I basically found this as I updated my instance and got a "Daemons not running" warning post update.

You started the daemons in your last comment with this command:

sudo -En -- ./bin/phd start

Should I be doing that too? I've been just doing /bin/phd start

I was just doing that to make them switch, since I was logged in as epriestley instead of root, so without it they would not have needed to switch. Equivalently, I can do this (as root):

root@orbital /Users/epriestley/dev/core/lib/phabricator $ PHACILITY_INSTANCE=local ./bin/phd start
Freeing active task leases...
Freed 1 task lease(s).
Preparing to launch daemons.
NOTE: Logs will appear in '/Users/epriestley/dev/core//log/local/phd/daemons.log'.

Starting daemons as epriestley
Launching daemon "PhabricatorRepositoryPullLocalDaemon".
Starting daemons as epriestley
Launching daemon "PhabricatorGarbageCollectorDaemon".
Starting daemons as epriestley
Launching daemon "PhabricatorTaskmasterDaemon".
Done.
root@orbital /Users/epriestley/dev/core/lib/phabricator $ PHACILITY_INSTANCE=local ./bin/phd status
ID Host      PID   Started                 Daemon                               Arguments
27 localhost 37988 Dec 23 2014, 1:34:23 PM PhabricatorRepositoryPullLocalDaemon          
28 localhost 37992 Dec 23 2014, 1:34:23 PM PhabricatorGarbageCollectorDaemon             
29 localhost 37999 Dec 23 2014, 1:34:23 PM PhabricatorTaskmasterDaemon

...which also works fine. (The PHACILITY_INSTANCE stuff is required by some local junk, but should be unrelated.)

When you start the daemons as btrahan it'll try to do a sudo -u _phd .... which you might not have the permissions for.
Normally this should result in an exception and the code will start the daemon as your normal user as a fallback.
If your sudo fails without the "CommandException" phd will think the daemons have started normally.
Can you try to manually run:
sudo -En -u _phd -- scripts/daemon/phd-daemon 'PhabricatorTaskmasterDaemon' --trace --verbose --phd='/var/tmp/phd/pid'
from your phabricator dir?

Looking at the code, despite the daemons not starting correctly, a command exception must not be what is being thrown.

13:41:49 ~/Dropbox/code
~> whoami
btrahan
13:41:50 ~/Dropbox/code
~> sudo su _phd
13:41:53 ~/Dropbox/code
~> whoami
btrahan
13:41:54 ~/Dropbox/code
~> sudo su root
13:41:57 /Users/btrahan/Dropbox/code
~> whoami
root
13:41:58 /Users/btrahan/Dropbox/code
~> sudo su btrahan
13:42:00 ~/Dropbox/code
~> whoami
btrahan
13:42:01 ~/Dropbox/code
~> sudo su asdsadsasdsadsadsasadasdasd
su: unknown login: asdsadsasdsadsadsasadasdasd

These sequence of commands show that I can 'su' to other user accounts except the _phd account. The _phd account does exist as far as I know.

@fabe - ran that command for you...

13:45:28 ~/Dropbox/code/phalanx (T5402)
~> sudo -En -u _phd -- scripts/daemon/phd-daemon 'PhabricatorTaskmasterDaemon' --trace --verbose --phd='/var/tmp/phd/pid'
Bringing daemon 'PhabricatorTaskmasterDaemon' online...
[2014-12-23 13:45:29] EXCEPTION: (FilesystemException) Requested path `/private/var/tmp/phd/pid' is not writable. at [<phutil>/src/filesystem/Filesystem.php:1043]
  #0 Filesystem::assertWritable(string) called at [<phutil>/src/filesystem/Filesystem.php:73]
  #1 Filesystem::assertWritableFile(string) called at [<phutil>/src/filesystem/Filesystem.php:89]
  #2 Filesystem::writeFile(string, string) called at [<phutil>/src/daemon/PhutilDaemonOverseer.php:159]
  #3 PhutilDaemonOverseer::__construct(array) called at [<phabricator>/scripts/daemon/launch_daemon.php:26]

What's the exit status of that command? That is, if you run echo $? afterward, what does it print?

This command, specifically:

sudo -En -u _phd -- scripts/daemon/phd-daemon 'PhabricatorTaskmasterDaemon' --trace --verbose --phd='/var/tmp/phd/pid'

From the code, I'd expect phd start to fail and then fall back to running as the current user. That's what it does on my machine if I force a (slightly different) failure.

Ok, i can reproduce it with a user who's shell is set to /sbin/nologin ...

Oh, the "slightly different" part matters. We try to do this write immediately after forking, which is why the failure is occurring.

Easiest fix miiiiiight just be to test that we can write to the file before we fork.

Huh, looks like the command output has changed. I probably had something funky from all the sudo su stuff... updated output with echo $?

13:52:45 ~/Dropbox/code/phalanx (T5402)
~> sudo -En -u _phd -- scripts/daemon/phd-daemon 'PhabricatorTaskmasterDaemon' --trace --verbose --phd='/var/tmp/phd/pid'
sudo: a password is required
13:52:48 ~/Dropbox/code/phalanx (T5402)
~> echo $?
0

epriestley mentioned this in D11040: Check daemon piddir writability before forking in daemons.Dec 23 2014, 10:07 PM

Yep works fine now..

epriestley mentioned this in rPHU92aefeccc0fd: Check daemon piddir writability before forking in daemons.Dec 23 2014, 10:10 PM

Hrrm, that sudo behavior is also broken. Looks like sudo -n is just messed up on OSX. I get this on a Linux system:

[dweller@ip-10-161-81-110 ec2-user] $ sudo -En -u root -- ls
sudo: a password is required
[dweller@ip-10-161-81-110 ec2-user] $ echo $?
1

Here's someone else hitting a similar issue:

https://github.com/mitmproxy/mitmproxy/issues/270

I guess the "fix" is to string-match stderr, like they do. :/

epriestley mentioned this in D11041: Detect goofy `sudo -n` output under OSX.Dec 23 2014, 10:21 PM

epriestley mentioned this in rPb3394c53d8e9: Detect goofy `sudo -n` output under OSX.Dec 23 2014, 10:45 PM

btrahan accepted this commit.Dec 24 2014, 7:02 PM

Changes (7)

Path

Size

resources/

sql/

autopatches/

20141223.daemonloguser.sql

src/

applications/

config/

check/

PhabricatorSetupCheckDaemons.php

daemon/

controller/

PhabricatorDaemonLogViewController.php

event/

PhabricatorDaemonEventListener.php

management/

PhabricatorDaemonManagementDebugWorkflow.php

PhabricatorDaemonManagementWorkflow.php

storage/

PhabricatorDaemonLog.php

rP2fedb6f941d8

View Options

resources/sql/autopatches/20141223.daemonloguser.sql

View Options

src/applications/config/check/PhabricatorSetupCheckDaemons.php

View Options

src/applications/daemon/controller/PhabricatorDaemonLogViewController.php

View Options

src/applications/daemon/event/PhabricatorDaemonEventListener.php

View Options

src/applications/daemon/management/PhabricatorDaemonManagementDebugWorkflow.php

View Options

src/applications/daemon/management/PhabricatorDaemonManagementWorkflow.php

View Options

src/applications/daemon/storage/PhabricatorDaemonLog.php

Start phd daemons as the correctly configured user and refuse otherwiseAudited2fedb6f941d8Actions

Description

Details

Event Timeline

Changes (7)

rP2fedb6f941d8

resources/sql/autopatches/20141223.daemonloguser.sql

src/applications/config/check/PhabricatorSetupCheckDaemons.php

src/applications/daemon/controller/PhabricatorDaemonLogViewController.php

src/applications/daemon/event/PhabricatorDaemonEventListener.php

src/applications/daemon/management/PhabricatorDaemonManagementDebugWorkflow.php

src/applications/daemon/management/PhabricatorDaemonManagementWorkflow.php

src/applications/daemon/storage/PhabricatorDaemonLog.php

Start phd daemons as the correctly configured user and refuse otherwise
Audited2fedb6f941d8
Actions