Page MenuHomePhabricator

Add a "Run Tests" Harbormaster build step

Authored by hach-que on Feb 5 2015, 2:08 PM.



Ref T5821. This adds a "Run Tests" build step to Harbormaster. With the "json-realtime" option, this enables Harbormaster to display test results in a format similar to Differential as soon as each test finishes (assuming the test engine supports it).

This doesn't yet work on Windows (it assumes a UNIX shell for running arc unit).

Test Plan

Ran a build plan against Arcanist itself and saw some pretty results:

Diff Detail

rP Phabricator
Lint OK
Unit Tests OK
Build Status
Buildable 4349
Build 4362: [Placeholder Plan] Wait for 30 Seconds

Event Timeline

hach-que updated this revision to Diff 28134.Feb 5 2015, 2:08 PM
hach-que retitled this revision from to Add a "Run Tests" Harbormaster build step.
hach-que updated this object.
hach-que edited the test plan for this revision. (Show Details)
hach-que added a reviewer: epriestley.

Some notes on this:

  • I'm not really happy with the amount of copy-pasta I had to do to bring across the Differential-styled table. I tried with a normal table and PHUI icons but it just looked like junk in comparison (and it was inconsistent). I don't know whether there's really a way to generalize the logic here either, as this has slightly different behaviour in how it shows information (e.g. it doesn't hide pass results by default, and I had to change $userdata = idx($test, 'userdata'); to $userdata = idx($test, 'userData'); to account for the different casing of userData).
  • While it does show results as they're reported, it doesn't yet show tests that are currently running. The diffs related to Arcanist need to be updated to emit JSON blobs when tests being (presumably with a result of "pending" or some such), after which we could update this to handle the new functionality.
  • There doesn't appear to be any way of adding additional classes to PHUI boxes. For the moment I've just added CSS styles assuming that if the Harbormaster styles are present then we're rendering the Differential table inside Harbormaster, but it'd be nicer to have some way of explicitly designating that the Differential table should have those additional styles.
hach-que abandoned this revision.Jun 18 2015, 1:46 PM

This hasn't been used in any serious way in a production environment.