Page MenuHomePhabricator

Fix some issues with empty daemon autoscale pools
ClosedPublic

Authored by epriestley on Jun 13 2016, 8:49 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Apr 11, 9:08 AM
Unknown Object (File)
Tue, Apr 2, 1:00 PM
Unknown Object (File)
Tue, Apr 2, 5:31 AM
Unknown Object (File)
Mon, Apr 1, 6:46 AM
Unknown Object (File)
Sun, Mar 31, 8:56 PM
Unknown Object (File)
Jan 21 2024, 8:12 PM
Unknown Object (File)
Jan 16 2024, 12:46 AM
Unknown Object (File)
Dec 27 2023, 6:59 PM
Subscribers
None

Details

Summary

Ref T10811. I'm not sure this causes any real problems, but I caught a couple of errors in the production logs in this vein:

[13-Jun-2016 14:26:55 UTC] [2016-06-13 14:26:55] ERROR 8: Undefined index:  at [/core/lib/libphutil/src/daemon/PhutilDaemonOverseer.php:302]
[13-Jun-2016 14:26:55 UTC] arcanist(head=stable, ref.master=29839e8c72c5, ref.stable=7b0aac5c6f31), libcore(), phabricator(head=stable, ref.stable=b26ecb189e48), phutil(head=stable, ref.master=992abe4a420c, ref.stable=52748950bb36), services(head=stable, ref.master=5b51b63027b7, ref.stable=66cf59af4580)
[13-Jun-2016 14:26:55 UTC]   #0 PhutilDaemonOverseer::updateAutoscale() called at [<phutil>/src/daemon/PhutilDaemonOverseer.php:224]
[13-Jun-2016 14:26:55 UTC]   #1 PhutilDaemonOverseer::run() called at [<phabricator>/scripts/daemon/launch_daemon.php:20]
[13-Jun-2016 14:26:55 UTC] [2016-06-13 14:26:55] EXCEPTION: (InvalidArgumentException) Argument 1 passed to PhutilDaemonOverseer::getAutoscaleProperty() must be an instance of PhutilDaemonHandle, null given, called in /core/lib/libphutil/src/daemon/PhutilDaemonOverseer.php on line 303 and defined at [<phutil>/src/error/PhutilErrorHandler.php:200]
[13-Jun-2016 14:26:55 UTC] arcanist(head=stable, ref.master=29839e8c72c5, ref.stable=7b0aac5c6f31), libcore(), phabricator(head=stable, ref.stable=b26ecb189e48), phutil(head=stable, ref.master=992abe4a420c, ref.stable=52748950bb36), services(head=stable, ref.master=5b51b63027b7, ref.stable=66cf59af4580)
[13-Jun-2016 14:26:55 UTC]   #0 PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phutil>/src/daemon/PhutilDaemonOverseer.php:278]
[13-Jun-2016 14:26:55 UTC]   #1 PhutilDaemonOverseer::getAutoscaleProperty(NULL, string, integer) called at [<phutil>/src/daemon/PhutilDaemonOverseer.php:303]
[13-Jun-2016 14:26:55 UTC]   #2 PhutilDaemonOverseer::updateAutoscale() called at [<phutil>/src/daemon/PhutilDaemonOverseer.php:224]
[13-Jun-2016 14:26:55 UTC]   #3 PhutilDaemonOverseer::run() called at [<phabricator>/scripts/daemon/launch_daemon.php:20]

These are caused by attempting to update empty autoscale pools. Currently, autoscale pools read configuration from the first daemon in the pool, but obviously that doesn't work if the pool is empty.

This could occur when shutting down or possibly while reloading daemons.

Also, add a diagnostic message bout pool autoscaling. This might be a little too chatty, but maybe useful for some other issues.

Test Plan
  • A reliable way to hit this was bin/phd debug task --autoscale, then ^C it. It no longer errors.
  • I used bin/worker flood to artificially create a lot of work, and saw the pool autoscale up and down properly.
  • I launched daemons normally (no autoscale).
  • I faked daemons as busy and saw the pool autoscale to full.
  • I changed config and saw daemons automatically reload properly.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Fix some issues with empty daemon autoscale pools.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
chad edited edge metadata.
This revision is now accepted and ready to land.Jun 13 2016, 8:50 PM
This revision was automatically updated to reflect the committed changes.