For a given revision, we want to run a bunch of tests on our CI infrastructure and report the results of those tests back on the revision page.
For example, we want to know the following things about a patch when it is applied to our project:
- Does the project compile?
- Do unit tests pass?
- Do integration tests pass?
Each one of these can be tested and reported on as the results become available.
It's currently possible to report on this stuff for a revision by creating a [[ https://secure.phabricator.com/book/phabricator/article/arcanist_lint_unit/ | custom unit test engine ]] and adding [[ https://secure.phabricator.com/chatlog/channel/6/?at=187856 | "postponed" ]] test results for each of the things you want to report on.
This is quite heavy-handed, though.
It would be better to have a direct way to create and update test results on a revision after the revision has been created. Evan mentioned [[ https://secure.phabricator.com/chatlog/channel/6/?at=187839 | something to this effect on #phabricator ]].
I'll add one thing to Evan's comments there, which is that it would be useful to be able to create a test before the results are available. That way reviewers can see that such and such tests are still running and their results are pending.