Root Problem: When one our builders gets behind for some reason then the order which builds happen is quite unpredictable. This is annoying for users submitting differentials as they end up waiting a long time for the CI to build their revision. It would be better if there was a predictable order which favoured builds which have been waiting longer.
What I think is happening is that once a build is triggered, it continually tries to acquire the builder every 15 seconds. Once a builder becomes available, the first build which tries to acquire the builder gets it, regardless of how long it has been waiting. A build queue would be more suitable for our purposes.