Page MenuHomePhabricator

Allow unit tests to be designated with an owner, so Phabricator can alert the right people
Open, Needs TriagePublic

Description

We have a large piece of software that multiple teams work on. Sometimes developers write tests that appear to pass, but fail randomly because of variations in the environment; often these tests are vertical or functional tests.

When these tests fail, the only person who gets notified is the person who committed the change. However, for these kinds of tests it's possible that it's just a test that's flaky rather than a failure based on the change that person has made (especially if it passes on the review build, but fails on the commit build).

Currently the person who committed the change has to look at the build, find the test that failed, and then attempt to either figure out who owns it (it's usually pretty identifiable that it's a flaky test), or in the more common circumstance, send an email around to the entire floor asking who owns the test and for someone to take ownership to fix it's flakiness.

We'd like to be able to designate owners for tests via the data we submit into Arcanist from our test runner. The expectation is that when a test fails, the owner of the test is notified as well as the person whose build failed. This would allow us to automatically have the right people notified when a flaky test has been written, but also in the event that a test is legitimately failing, it would allow the owners of that section of the code base to be notified so that they can ensure the issue is resolved by the person who introduced the issue.