Page MenuHomePhabricator

After hard non-graceful shutdown, Phabricator insists old daemons are still running.
Closed, ResolvedPublic

Description

After hard non-graceful shutdown (of the host computer), Phabricator insists old daemons are still running. I've attempted force stopping and restarting, to no avail. I've verified that the PID files no longer exist.
I've even attempted (in a not very good way) to create PID files for them, but only to the extent of copying a random PID file for a different daemon and changing the PID inside.
Since Phabricator still thinks the daemons are running, and can't update their configuration, it's spamming me with the "You have unresolved setup issues..." banner.
I haven't been able to find any proper way to purge them properly (or at all). This should probably be a thing.

Event Timeline

grobe0ba raised the priority of this task from to Needs Triage.
grobe0ba updated the task description. (Show Details)
grobe0ba added a subscriber: grobe0ba.

We don't look for PID files or examine the process table because:

  • the webserver may not have permission to read them;
  • the daemons may be running on a different machine.

Instead, daemons periodically connect to the server and tell it that they're still running.

Daemons should move to an "unknown" state automatically after 6 minutes (indicating that they are likely not running), and be automatically removed (assumed dead) after 1 hour.

At HEAD, the "You have unresolved setup issues..." warning should only check active daemons, so it should disappear after 6 minutes. If you're running an older version of the code (before D12088 / rPbc28b2c3 / March 15), we also checked the "unknown" daemons, so the setup warning potentially took an hour to disappear.

Is this consistent with what you're seeing, or are you seeing different behavior?

I'm seeing somewhat different behaviour. I can't be 100% sure when exactly it finally figured out that those daemons weren't coming back, because I sleep during the day, but it took at least 4 hours (since I was still using it during that period). When I refreshed everything just now, the banner went away and everything appears to be chooching along like it should be.

Hmm, it would also appear that I desperately need to update. I was unaware that the FreeBSD port maintainer hasn't updated the port since summer of last year. My apologies for probably wasting everyone's time.

epriestley claimed this task.

Ah, yeah, if you're far behind HEAD you probably still had the 1 hour behavior. I'd expect that updating will improve things. Let us know if you run into anything else.