Add configuration to specify additional bots/daemons on phd start
Closed, WontfixPublic

Description

7:35 PM <•epriestley> I guess there are two reasonable generalizations:
7:35 PM <tberman> it can check running pids as it stops?
7:35 PM <•epriestley> - Add some config to say "launch bots using these config files on 'phd start'"
7:35 PM <tberman> yeah, doesnt that exist for # of taskmasters as well?
7:36 PM <•epriestley> - Add some config to say "launch these extra daemons on 'phd start'"
7:36 PM <•epriestley> Yeah, there's a taskmaster count config already
7:36 PM <tberman> generalize that into a phd start runlist
7:36 PM <•epriestley> I have a minor inclination toward the bot-specific list I guess?
7:37 PM <•epriestley> But if you want to write either one I'd be happy to accept it.
7:37 PM <•epriestley> Or you can file a task and I'll write the first one, probably, eventually. :)
7:37 PM <tberman> ill look at it
7:37 PM <tberman> ill do that for sure
7:37 PM <tberman> its less of a 'must scratch' itch as getting phabot in hipchat
7:37 PM <•epriestley> (The bot-specific list is selfishly nice since we can sanity check it more and provide better errors.)
7:37 PM <•epriestley> yeah

tberman added a subscriber: tberman.
tberman added a subscriber: epriestley.
epriestley triaged this task as Low priority.May 9 2013, 12:25 PM

I have a similar request. Personally, I feel that the phd.start-taskmasters configuration could be replaced with a more general phd.start-daemons configuration, which would list all of the daemons that are to be launched on ./bin/phd start.

The default configuration would be as follows:

{
  "PhabricatorGarbageCollectorDaemon"    : 1,
  "PhabricatorRepositoryPullLocalDaemon" : 1,
  "PhabricatorTaskmasterDaemon"          : 4
}

That wouldn't be useful for starting the chat bot (it takes a mandatory argument), and I think all cases for starting anything other than 1 GC or 1 PullLocal daemon are conditional (based on the host).

What's the issue you're hitting, specifically?

I'm not hitting an issue, I would just like a more sane way to tell Phabricator what daemons should be launched on phd start.

An alternative format could be used for passing arguments perhaps:

{
  "PhabricatorGarbageCollectorDaemon"    : 1,
  "PhabricatorRepositoryPullLocalDaemon" : [
    ["X", "Y"],
    ["X", "--not", "Y", "Z"]
  ],
  "PhabricatorTaskmasterDaemon"          : 4,
  "PhabricatorBot"                       : [
    ["config1.json"],
    ["config2.json"]
  ]
}

◀ Merged tasks: T4181.

joshuaspence edited this Maniphest Task.May 18 2014, 3:47 AM
joshuaspence removed joshuaspence as the assignee of this task.Jun 10 2014, 7:02 AM
chad changed the visibility from "All Users" to "Public (No Login Required)".Aug 4 2015, 7:05 PM
eadler added a project: Restricted Project.Mar 28 2016, 8:32 PM
eadler moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Mar 28 2016, 8:32 PM
epriestley moved this task from Backlog to Availability on the Daemons board.Apr 8 2016, 8:39 PM
epriestley closed this task as Wontfix.Apr 8 2016, 9:20 PM
epriestley claimed this task.

You can now start daemons at a low level by piping JSON into scripts/daemon/phd-daemon:

{
  "daemonize": true,
  "log": "/Users/epriestley/dev/core/log/local/phd/daemons.log",
  "piddir": "/Users/epriestley/dev/core/run/local/phd",
  "daemons": [
    {
      "class": "PhabricatorRepositoryPullLocalDaemon"
    },
    {
      "class": "PhabricatorTriggerDaemon"
    },
    {
      "class": "PhabricatorTaskmasterDaemon",
      "autoscale": {
        "group": "task",
        "pool": 4,
        "reserve": 0
      }
    }
  ]
}

This is some undocumented, sketchy magic, but is unlikely to change much in the future. However, any variant of this puts you at risk of not running new daemons.

More broadly:

  • I expect to remove the --not, and <repository> flags from PullLocal daemon soon and just make it use the repository cluster definition, removing any motivation for starting it with special flags.
  • I expect we'll eventually make bots first-class and start them as part of the rest of the daemon group (although not for a long time).
  • You can accomplish the same thing with daemons.sh instead of daemons.json, by having it run bin/phd stop, then a sequence of start and launch commands.