Page MenuHomePhabricator
Create Task
Maniphest T12331

Taskmasters on `admin.phacility.com` are not scaling up correctly
Closed, ResolvedPublic
Actions

Description

After the recent deploy, the admin taskmasters don't appear to be scaling up properly. This sometimes leads to queue backlogs, particularly when a large backup task blocks progress.

Revisions and Commits

Restricted Differential RevisionRestricted Diffusion Commit
Restricted Differential RevisionRestricted Diffusion Commit
Restricted Differential RevisionRestricted Diffusion Commit
Restricted Differential RevisionRestricted Diffusion Commit
Restricted Differential RevisionRestricted Diffusion Commit
Restricted Differential RevisionRestricted Diffusion Commit

Event Timeline

epriestley created this task.Feb 28 2017, 12:06 PM
Herald added subscribers: chad, eadler. · View Herald TranscriptFeb 28 2017, 12:06 PM
epriestley added a revision: Restricted Differential Revision.Feb 28 2017, 3:31 PM
epriestley added a revision: Restricted Differential Revision.Feb 28 2017, 3:37 PM
epriestley added a comment.Feb 28 2017, 3:43 PM

I'm picking those to stable and sending them to production now.

epriestley added a commit: Restricted Diffusion Commit.Feb 28 2017, 3:43 PM
epriestley added a commit: Restricted Diffusion Commit.
epriestley added a commit: Restricted Diffusion Commit.
epriestley added a commit: Restricted Diffusion Commit.
epriestley added a revision: Restricted Differential Revision.Feb 28 2017, 4:10 PM
epriestley added a comment.Feb 28 2017, 4:14 PM

I'm picking D17433 now and sending it to admin.

epriestley added a commit: Restricted Diffusion Commit.Feb 28 2017, 4:15 PM
epriestley added a commit: Restricted Diffusion Commit.
epriestley added a comment.Feb 28 2017, 4:21 PM

The pool appears to be scaling up and down properly now. I'm going to deploy the fix to the secure and repo tiers since it may affect all instances.

epriestley closed this task as Resolved.Feb 28 2017, 4:37 PM
epriestley claimed this task.

I've deployed everything so I believe this is now resolved. I'll keep an eye on it going forward.

Phabricator · Built by Phacility