Add a `bin/herald test ...` for doing test runs via the CLI
ClosedPublic
Actions

Authored by epriestley on Nov 14 2018, 11:29 PM.

Details

Reviewers

amckinley

Maniphest Tasks

T13216: 2018 Week 45-47 Bonus Content

Commits

rP533e4e13b35a: Add a `bin/herald test ...` for doing test runs via the CLI

Summary

Ref T13216. See D19666. It's currently tricky to profile Herald test runs since you have to submit a form and repeating them is a bit of a mess. Provide a simple CLI wrapper so we can use --xprofile. This is also maybe nice-to-have if we're ever debugging anything here.

Test Plan

Ran bin/herald test --object ... --type ... and got a sensible looking transcript in the UI.

Diff Detail

Repository

rP Phabricator

Branch

herald3

Lint

Lint Passed

Unit

Tests Passed

Build Status

Buildable 21148
Build 28752: Run Core Tests
Build 28751: arc lint + arc unit

Event Timeline

epriestley created this revision.Nov 14 2018, 11:29 PM

Harbormaster returned this revision to the author for changes because remote builds failed.Nov 14 2018, 11:29 PM

Harbormaster failed remote builds in B21148: Diff 47299!

Stacked on qsprintf(),.

amckinley accepted this revision.Nov 15 2018, 10:20 PM

amckinley added inline comments.

src/applications/herald/management/HeraldTestManagementWorkflow.php
40	Maybe we want to take a PHID as an argument in addition to an object name?
114	It might make sense to test a disabled Herald rule, since the rule might be disabled for bugginess but someone wants to test it from here before reenabling it.

This revision is now accepted and ready to land.Nov 15 2018, 10:20 PM

epriestley marked 2 inline comments as done.Nov 15 2018, 11:21 PM

epriestley added inline comments.

src/applications/herald/management/HeraldTestManagementWorkflow.php
40	`withNames()` is roughly `withMonogramsOrPHIDs(...)` and will load objects by PHID if PHIDs are passed in.
114	We could theoretically offer single-rule test runs but it would be slightly tricky because of the "Another Herald Rule: ..." condition, which can mean that individual rules don't truly run in isolation. Reasonable-ish workarounds today are to remove the actions or add an impossible condition at the end of the rule you want to test.