Page MenuHomePhabricator
Create Task
Maniphest T10559

Lease Working Copy Hangs on lease failure, but should fail
Closed, ResolvedPublic
Actions

Description

When trying to acquire a lease in drydock that fails for any reason the Lease Working Copy step will hang.

Reproduce:

  1. Modify the blueprint in someway that makes it fail. I came across this on accident by adding drydockcommandinterface::INTERFACE_TYPE; to activateLease on accident. note the all lowercase letters.
  2. Start any build that uses the Lease Working Copy build step.

Screen Shot 2016-03-10 at 4.29.36 PM.png (948×2 px, 303 KB)

The failure in this picture happened instantly upon trying to create a lease.

Version: 2c67d9c

Related Objects

Event Timeline

michaeljs1990 created this task.Mar 10 2016, 9:34 PM
michaeljs1990 updated the task description. (Show Details)
michaeljs1990 updated the task description. (Show Details)Mar 10 2016, 9:44 PM
michaeljs1990 updated the task description. (Show Details)Mar 10 2016, 11:14 PM
joshuaspence added a subscriber: joshuaspence.Mar 11 2016, 8:26 AM
epriestley renamed this task from Lease Working Copy Hangs to Lease Working Copy Hangs on lease failure, but should fail.Mar 14 2016, 11:57 PM
epriestley added a subscriber: epriestley.

(I think that title edit is accurate, let me know if I misunderstood.)

This is probably fairly easy to fix. I made some small inroads on cleaning up some of this stuff recently, but haven't gotten quite as far as the Drydock/Harbormaster boundary.

chad added a project: Drydock.May 2 2016, 7:55 PM
epriestley mentioned this in T13073: Plans: Drydock for normal software use cases where builds take more than 45 seconds.Feb 12 2018, 11:17 PM
epriestley closed this task as Resolved.Feb 13 2018, 4:03 PM
epriestley claimed this task.

After changes T13073, I am no longer able to reproduce this. That task has made some improvements and may have fixed whatever happened here.

My test case was to add $x = drydockcommandinterface::xyz; to activateLease(), then activate a lease with bin/drydock lease. This broke and destroyed the lease, which is currently the expected behavior.

Note that some types of errors can instantly fatal the process in a way that we can not recover from. In an extreme case, if you posix_kill(getmypid(), 9) to immediately SIGKILL yourself, we can't clean up or recover anything. The lease will still move forward eventually, but it will likely take an hour for the task to timeout and retry.

Our handling of internal PHP errors, especially between PHP5 vs PHP7, has changed over time, and it's possible that this got more-severe error handling when it was originally filed, more similar to killing itself than to throwing.

Phabricator · Built by Phacility