Page MenuHomePhabricator

Support daemon autoscaling in libphutil
ClosedPublic

Authored by epriestley on Feb 23 2015, 12:06 AM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Sep 5, 5:35 PM
Unknown Object (File)
Sat, Aug 31, 12:51 AM
Unknown Object (File)
Thu, Aug 29, 11:15 AM
Unknown Object (File)
Wed, Aug 28, 10:22 PM
Unknown Object (File)
Aug 28 2024, 1:41 AM
Unknown Object (File)
Aug 26 2024, 4:04 PM
Unknown Object (File)
Aug 22 2024, 7:41 PM
Unknown Object (File)
Aug 15 2024, 5:39 AM
Subscribers

Details

Summary

Ref T7352. Autoscaling allows daemons (particularly, the taskmaster daemon) to scale up (launch more copies) or down (have some copies exit and not restart).

The primary goal is to improve resource utilization in the cluster. In particular:

  • Instances with relatively little activity will automatically scale down to 1 taskmaster.
  • Instances with a lot of activity can scale up to some higher-than-average ceiling.

In the cluster, compared to the current strategy of a fixed number of taskmasters per isntance, this uses less resources in all cases except when the daemons are fully loaded (when it uses the same amount of resources), and allows us to allocate resources much more efficiently.

This doesn't really impact normal installs much. They'll still get autoscaling, but the memory footprint of daemons isn't large enough to matter in most cases.

This (and the next diff) don't actually autoscale taskmasters yet, they just build all the autoscaling infrastructure.

Test Plan
  • Ran phd debug --autoscale intensity.
  • Saw pool scale up to 8 daemons on odd minutes, launching one new daemon every few seconds, then stabilize.
  • Saw the pool scale down to 1 daemon at 15 seconds past the minute on even minutes. The pool would normally scale less abruptly, but the busy-ness function for the HighIntensity daemon means this is the expected behavior.

Diff Detail

Repository
rPHU libphutil
Branch
daemon5
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 4611
Build 4625: [Placeholder Plan] Wait for 30 Seconds

Event Timeline

epriestley retitled this revision from to Support daemon autoscaling in libphutil.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: btrahan.
btrahan edited edge metadata.
This revision is now accepted and ready to land.Feb 23 2015, 6:04 PM
This revision was automatically updated to reflect the committed changes.