Page MenuHomePhabricator

Remove Drydock host resource limits and give working copies simple limits
ClosedPublic

Authored by epriestley on Oct 25 2015, 6:12 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Dec 15, 9:13 PM
Unknown Object (File)
Thu, Dec 5, 4:33 PM
Unknown Object (File)
Mon, Dec 2, 10:39 AM
Unknown Object (File)
Wed, Nov 27, 5:06 PM
Unknown Object (File)
Wed, Nov 27, 9:23 AM
Unknown Object (File)
Wed, Nov 27, 6:01 AM
Unknown Object (File)
Nov 8 2024, 12:46 PM
Unknown Object (File)
Nov 8 2024, 12:46 PM
Subscribers
None

Details

Summary

Ref T9252. Right now, we have very strict limits on Drydock: one lease per host, and one working copy per working copy blueprint.

These are silly and getting in the way of using "Land Revision" more widely, since we need at least one working copy for each landable repository.

For now, just remove the host limit and put a simple limit on working copies. This might need to be fancier some day (e.g., limit working copies per-host) but it is generally reasonable for the use cases of today.

Also add a --background flag to make testing a little easier.

(Limits are also less important nowadays than they were in the past, because pools expand slowly now and we seem to have stamped out all the "runaway train" bugs where allocators go crazy and allocate a million things.)

Test Plan
  • With a limit of 5, ran 10 concurrent builds and saw them finish after allocating 5 total resources.
  • Removed limit, raised taskmaster concurrency to 128, ran thousands of builds in blocks of 128 or 256.
    • Saw Drydock gradually expand the pool, allocating a few more working copies at first and a lot of working copies later.
    • Got ~256 builds in ~140 seconds, which isn't a breakneck pace or anything but isn't too bad.
    • This stuff seems to be mostly bottlenecked on sbuild throttling inbound SSH connections. I haven't tweaked it.

Diff Detail

Repository
rP Phabricator
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Remove Drydock host resource limits and give working copies simple limits.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
This revision is now accepted and ready to land.Oct 26 2015, 7:27 PM
This revision was automatically updated to reflect the committed changes.