Page MenuHomePhabricator

Display when builds happen with status somewhere handy, like the differential timeline
Closed, ResolvedPublic


Harbormaster actions (such as the build passing) used to be part of the differential timeline:

What I am doing a now is a lot of testing and re-starting builds. Form differential the only way to tell that a new build finished is that the artifact url (T8659) has changed. The buildable itself has a timeline, but that just shows the most recent status and when I restarted it, but not the history of pass/fails. This is moderately annoying when debugging builds and restarting them a lot, although I do not expect that to be the common case.

I'm worried about cases like:

  • R: Hey the code looks okay but why does test X fail?
  • A: (hits button) No they pass.
  • R: ¯\_(ツ)_/¯


  • R: Please fix x.
  • A: (fixes x, some flakey test fails)
  • R: Wait did the tests fail before?

But I have not used Harbormaster enough 'for real' to run into those. I'm also unsure if the 'just put it in the timeline' approach is viable when a diff kicks off multiple builds.

Related chatter:

Event Timeline

cburroughs updated the task description. (Show Details)
cburroughs added a subscriber: cburroughs.

Some additional discussion in T8650.

Harbormaster should show buildable history better, although I also expect restarts to be rare in "reasonable" test suites.

(This may be naive, and possibly we should assume that a large fraction of builds in real-world scenarios produce different results when repeated and that restarts will be common, but I feel like we have a stronger moral position here than on other philosophical issues because there's more general agreement that flaky tests which depend on flaky external services are bad.)

epriestley edited projects, added Harbormaster (v3); removed Harbormaster.
eadler added a project: Restricted Project.Aug 5 2016, 4:43 PM

This should work again now. The behavior probably needs refinement -- I don't remember exactly where we left it and I think there was some disagreement about who should or shouldn't receive build failure notifications before D10911 accidentally disabled this, but I think T9892 and other Harbormaster tasks discuss most of that.