HomePhabricator

Support daemon autoscaling in libphutil

Description

Support daemon autoscaling in libphutil

Summary:
Ref T7352. Autoscaling allows daemons (particularly, the taskmaster daemon) to scale up (launch more copies) or down (have some copies exit and not restart).

The primary goal is to improve resource utilization in the cluster. In particular:

  • Instances with relatively little activity will automatically scale down to 1 taskmaster.
  • Instances with a lot of activity can scale up to some higher-than-average ceiling.

In the cluster, compared to the current strategy of a fixed number of taskmasters per isntance, this uses less resources in all cases except when the daemons are fully loaded (when it uses the same amount of resources), and allows us to allocate resources much more efficiently.

This doesn't really impact normal installs much. They'll still get autoscaling, but the memory footprint of daemons isn't large enough to matter in most cases.

This (and the next diff) don't actually autoscale taskmasters yet, they just build all the autoscaling infrastructure.

Test Plan:

  • Ran phd debug --autoscale intensity.
  • Saw pool scale up to 8 daemons on odd minutes, launching one new daemon every few seconds, then stabilize.
  • Saw the pool scale down to 1 daemon at 15 seconds past the minute on even minutes. The pool would normally scale less abruptly, but the busy-ness function for the HighIntensity daemon means this is the expected behavior.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11859

Details

Provenance
epriestleyAuthored on
epriestleyPushed on Feb 24 2015, 10:51 PM
Reviewer
btrahan
Differential Revision
D11859: Support daemon autoscaling in libphutil
Parents
rPHU0e5b0f293436: Receive most overseer configuration over stdin
Branches
Unknown
Tags
Unknown
Tasks
T7352: Improve daemon scalability in the cluster