Add a "Run Tests" Harbormaster build step
AbandonedPublic
Actions

Authored by hach-que on Feb 5 2015, 2:08 PM.

Details

Reviewers

epriestley
chad

Group Reviewers

Blessed Reviewers

Maniphest Tasks

T5821: Implement "Run Arc Unit" build step and support showing unit test results inside Harbormaster

Summary

Ref T5821. This adds a "Run Tests" build step to Harbormaster. With the "json-realtime" option, this enables Harbormaster to display test results in a format similar to Differential as soon as each test finishes (assuming the test engine supports it).

This doesn't yet work on Windows (it assumes a UNIX shell for running arc unit).

Test Plan

Ran a build plan against Arcanist itself and saw some pretty results:

Diff Detail

Repository

rP Phabricator

Branch

harbormaster-unit

Lint

Lint Passed

Unit

Tests Passed

Build Status

Buildable 4349
Build 4362: [Placeholder Plan] Wait for 30 Seconds

Event Timeline

hach-que updated this revision to Diff 28134.Feb 5 2015, 2:08 PM

hach-que retitled this revision from to Add a "Run Tests" Harbormaster build step.

hach-que updated this object.

hach-que edited the test plan for this revision. (Show Details)

hach-que added a reviewer: epriestley.

hach-que added a task: T5821: Implement "Run Arc Unit" build step and support showing unit test results inside Harbormaster.

Herald added reviewers: chad, Blessed Reviewers. · View Herald TranscriptFeb 5 2015, 2:08 PM

Herald added subscribers: epriestley, Korvin. · View Herald Transcript

Some notes on this:

I'm not really happy with the amount of copy-pasta I had to do to bring across the Differential-styled table. I tried with a normal table and PHUI icons but it just looked like junk in comparison (and it was inconsistent). I don't know whether there's really a way to generalize the logic here either, as this has slightly different behaviour in how it shows information (e.g. it doesn't hide pass results by default, and I had to change $userdata = idx($test, 'userdata'); to $userdata = idx($test, 'userData'); to account for the different casing of userData).
While it does show results as they're reported, it doesn't yet show tests that are currently running. The diffs related to Arcanist need to be updated to emit JSON blobs when tests being (presumably with a result of "pending" or some such), after which we could update this to handle the new functionality.
There doesn't appear to be any way of adding additional classes to PHUI boxes. For the moment I've just added CSS styles assuming that if the Harbormaster styles are present then we're rendering the Differential table inside Harbormaster, but it'd be nicer to have some way of explicitly designating that the Differential table should have those additional styles.