Page MenuHomePhabricator

When a Drydock Blueprint promises us a resource but can't deliver, continue believing in it
ClosedPublic

Authored by epriestley on Feb 13 2018, 11:31 AM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Apr 25, 12:08 AM
Unknown Object (File)
Fri, Apr 19, 2:49 AM
Unknown Object (File)
Thu, Apr 11, 8:24 AM
Unknown Object (File)
Sat, Mar 30, 6:48 PM
Unknown Object (File)
Sat, Mar 30, 6:48 PM
Unknown Object (File)
Sat, Mar 30, 6:48 PM
Unknown Object (File)
Sat, Mar 30, 6:48 PM
Unknown Object (File)
Mar 10 2024, 3:07 PM
Subscribers
Restricted Owners Package

Details

Summary

Ref T13073. When a Blueprint says it will be able to allocate a resource but then throws an exception while attempting that allocation, we currently fail the lease permanently.

This is excessively harsh. This blueprint may have the best of intentions and have encountered a legitimately unforseeable failure (like a vm.new call to build a VM failed) and be able to succeed in the future.

Even if this blueprint is a dirty liar, other blueprints (or existing resources) may be able to satisfy the lease in the future.

Even if every blueprint is implemented incorrectly, leaving the lease alive lets it converge to success after the blueprints are fixed.

Instead of failing, log the issue and yield.

(In the future, it might make sense to distinguish more narrowly between "actually, all the resources are used up" and all other failure types, since the former is likely more routine and less concerning.)

Test Plan
  • Wrote a broken Hoax blueprint which always claims it can allocate but never actually allocates (just throw in allocateResource()).
  • Used bin/phd drydock lease to acquire a Hoax lease.
  • Before patch: lease abruptly failed permanently.
  • After patch: lease yields after allocation fails.

Screen Shot 2018-02-13 at 3.29.49 AM.png (1×2 px, 240 KB)

Diff Detail

Repository
rP Phabricator
Lint
Lint Not Applicable
Unit
Tests Not Applicable