At the moment when we're building a diff, there's a multitude of ways that it can fail, in particular:
- Merge conflicts
- Build failure
- Unit test failure
It'd be preferable to expose this reason on the "Harbormaster failed to build <buildable>" transaction so that users know why the build failure occurred (without having to scan through logs).
I think this is dependent on supporting "Failure Steps" in build plans; that is, steps to run when a build fails. Then we have some sort of "Scan Log and Set Reason" build step that searches logs for a particular regex and sets the build failure reason if it matches.