Page MenuHomePhabricator
Create Task
Maniphest T9276

Display when builds happen with status somewhere handy, like the differential timeline
Closed, ResolvedPublic
Actions

Description

Harbormaster actions (such as the build passing) used to be part of the differential timeline: https://cloud.githubusercontent.com/assets/1963030/9352309/97d1ed7a-4660-11e5-91ad-41646222c850.png

What I am doing a now is a lot of testing and re-starting builds. Form differential the only way to tell that a new build finished is that the artifact url (T8659) has changed. The buildable itself has a timeline, but that just shows the most recent status and when I restarted it, but not the history of pass/fails. This is moderately annoying when debugging builds and restarting them a lot, although I do not expect that to be the common case.

I'm worried about cases like:

  • R: Hey the code looks okay but why does test X fail?
  • A: (hits button) No they pass.
  • R: ¯\_(ツ)_/¯

or

  • R: Please fix x.
  • A: (fixes x, some flakey test fails)
  • R: Wait did the tests fail before?

But I have not used Harbormaster enough 'for real' to run into those. I'm also unsure if the 'just put it in the timeline' approach is viable when a diff kicks off multiple builds.

Related chatter:

Revisions and Commits

rP Phabricator
AuditedD17139rPef05bf335d8e Allow Harbormaster builds to publish to a different object

Related Objects

Event Timeline

cburroughs created this task.Aug 27 2015, 6:52 PM
cburroughs updated the task description. (Show Details)
cburroughs added projects: Harbormaster, Differential.
cburroughs added a subscriber: cburroughs.
joshuaspence added a subscriber: joshuaspence.Aug 27 2015, 8:25 PM
richardvanvelzen added a subscriber: richardvanvelzen.Aug 30 2015, 3:29 PM
epriestley added a subscriber: epriestley.Aug 31 2015, 1:15 PM

Some additional discussion in T8650.

Harbormaster should show buildable history better, although I also expect restarts to be rare in "reasonable" test suites.

(This may be naive, and possibly we should assume that a large fraction of builds in real-world scenarios produce different results when repeated and that restarts will be common, but I feel like we have a stronger moral position here than on other philosophical issues because there's more general agreement that flaky tests which depend on flaky external services are bad.)

epriestley moved this task from Backlog to v2 on the Harbormaster board.Aug 31 2015, 1:15 PM
epriestley mentioned this in T10260: Harbormaster Herald Rules.Feb 3 2016, 5:31 PM
elesh added a subscriber: elesh.Feb 5 2016, 5:59 AM
avivey mentioned this in D15247: Herald Harbormaster Buildable Rules.Feb 12 2016, 6:48 PM
epriestley moved this task from v2 to v3 on the Harbormaster board.Mar 3 2016, 3:06 PM
epriestley edited projects, added Harbormaster (v3); removed Harbormaster.
jcarrillo7 added a subscriber: jcarrillo7.Jun 6 2016, 8:10 PM
Herald added a subscriber: eadler. · View Herald TranscriptJun 6 2016, 8:10 PM
eadler added a project: Restricted Project.Aug 5 2016, 4:43 PM
epriestley moved this task from Backlog to v3 (Infrastructure) on the Differential board.Dec 29 2016, 4:28 PM
epriestley added a revision: D17139: Allow Harbormaster builds to publish to a different object.Jan 4 2017, 9:24 PM
epriestley closed this task as Resolved by committing rPef05bf335d8e: Allow Harbormaster builds to publish to a different object.Jan 4 2017, 9:46 PM
epriestley added a commit: rPef05bf335d8e: Allow Harbormaster builds to publish to a different object.
epriestley added a comment.Jan 4 2017, 9:56 PM

This should work again now. The behavior probably needs refinement -- I don't remember exactly where we left it and I think there was some disagreement about who should or shouldn't receive build failure notifications before D10911 accidentally disabled this, but I think T9892 and other Harbormaster tasks discuss most of that.

Phabricator · Built by Phacility