Page MenuHomePhabricator

Adjust the Drydock allocator to limit each pending lease to one allocating resource
ClosedPublic

Authored by epriestley on May 9 2022, 5:24 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Apr 20, 5:56 PM
Unknown Object (File)
Sat, Apr 20, 4:58 PM
Unknown Object (File)
Fri, Apr 19, 8:57 AM
Unknown Object (File)
Wed, Apr 17, 2:57 PM
Unknown Object (File)
Mon, Apr 15, 10:48 PM
Unknown Object (File)
Thu, Apr 11, 8:16 AM
Unknown Object (File)
Mon, Apr 8, 8:32 AM
Unknown Object (File)
Sun, Apr 7, 12:08 AM
Subscribers
Restricted Owners Package

Details

Summary

Ref T13677. Currently, one lease may cause multiple resources to allocate simultaneously if it starts allocating one, then wakes up from a yield later on and still sees no available resources.

This is never desired -- or, at least, produces desirable behavior only entirely by accident. Normally, it causes an excess of resources to allocate.

This is not a catastrophic problem: the extra resources usually get used sooner or later or cleaned up; and the total amount of badness is limited by overall resource allocation limits.

However, this behavior is also suppressed by an artificial "25% of current pool size" growth limit throttle which I intend to remove. Removing this throttle without fixing the allocator behavior could make this "too many resources" problem worse.

Change the allocator so that a lease that has started allocating a resource won't allocate another resource until the first resource leaves the "pending" state.

This also fixes some general oddness with the allocator and attempts to simplify the structure.

Test Plan
  • Ran 8 taskmasters.
  • Destroyed all resources and leases.
  • Leased 4 working copies.
    • Saw exactly 4 resources build and lease, all simultaneously.
  • Destroyed all resources and leases.
  • Leased 32 working copies.
    • Saw exactly 32 resources build and lease, approximately 8 at a time (limited by taskmasters).
  • Destroyed all leases (but not resources).
  • Leased 32 working copies, saw them satisfied by existing resources.

Diff Detail

Repository
rP Phabricator
Branch
drydock-3
Lint
Lint Errors
Unit
Test Failures
Build Status
Buildable 25702
Build 35530: arc lint + arc unit

Unit TestsFailed

TimeTest
225 msPhabricatorLibraryTestCase::testLibraryMap
Assertion failed, expected 'true' (at PhutilLibraryTestCase.php:51): The library map is out of date. Rebuild it with `arc liberate`. These entries differ: class.DrydockLeaseWaitingForActivationLogType, class.DrydockLeaseWaitingForReclamationLogType, xmap.DrydockLeaseWaitingForActivationLogType, xmap.DrydockLeaseWaitingForReclamationLogType.
220 msPhabricatorCelerityTestCase::testCelerityMaps
3 assertions passed.
9 msPhabricatorConduitTestCase::testConduitMethods
1 assertion passed.
0 msPhabricatorInfrastructureTestCase::testApplicationsInstalled
1 assertion passed.
2 msPhabricatorInfrastructureTestCase::testRejectMySQLNonUTF8Queries
1 assertion passed.
View Full Test Results (1 Failed · 6 Passed)

Event Timeline

Owners added a subscriber: Restricted Owners Package.May 9 2022, 5:24 PM
  • Fix some variable names and exception handling.