Page MenuHomePhabricator

Create audit requests based on a package commits
Closed, ResolvedPublic

Description

  1. I expect that when a package is defined with Auditable enabled, each commit that affect the package will trigger an Audit rule.
  1. When adding Herald rule to create audit request to specific package (group of owners), the audit process is unexpected. If someone raises concern, there is no way to remove it. Moreover, if someone approves the commit, it still appears in a pending audit requests queue.
  1. Herald is lacking documentation. I find it impossible to configure the simple flow I want to work correctly.

I have several packages. On each commit to a package, I want that audit request would be issued to the relevant group of people. Tried to configure it using packages, projects, etc' and didn't find a working flow. There are some bugs (T10090 and T9430) that make it hard to debug these issues.

Event Timeline

eadler added a project: Restricted Project.Apr 11 2016, 9:41 PM

I expect that when a package is defined with Auditable enabled, each commit that affect the package will trigger an Audit rule.

This is not the expected behavior of auditing, although the details may not be documented anywhere. It is expected that commits do not trigger audits if any of these conditions are true:

  • The package is archived (after D15907).
  • A package owner authored or accepted the change.

You can use Herald to explicitly create audits outside of these rules if you have an unusual need for strong auditing rules. I will also document these rules.

Can you provide reproduction steps which exhibit a behavior contrary to this expectation?

If someone raises concern, there is no way to remove it.

The expectation is that the user who added the concern can remove it.

Can you provide reproduction steps which exhibit a behavior contrary to this expectation?

See also T2393.

Herald is lacking documentation. I find it impossible to configure the simple flow I want to work correctly.

Without more specifics, I don't know how to resolve this. The feedback we get from users about Herald is mostly questions about how to write complex/obscure rules, or requests to add new fields. The advanced nature of this feedback suggests that many users are currently able to get started with Herald.

Herald has documentation, available in the Help menu. Although it's a bit out of date, it looks like it still accurately describes the basic operation and behavior of Herald.


Beyond documenting the expected behavior of "Audit: Enabled", I am not sure how to move forward here. Points (1) and (2) appear to describe bugs, but need reproduction steps (see Providing Reproduction Steps and Contributing Bug Reports).

We are currently reworking some behaviors related to audit in connection with T10939. If you don't have a chance to provide feedback on this task before that work concludes, I'll close it after documenting the behavior of the "Auditing" setting. In this case, please file new tasks with reproduction instructions if you later want to report bugs.

If a package is defined with multiple owners (or a group ownership), it is a reasonable expectation that even when an owner commits a change, other owners will review that change. This is how owners has worked up until the T10939 changes recently.

T10939 describes a very complex and specific set of requirements, which I do not think can be met by the Owners implementation alone, either before or after the changes. But if you insist that the current behaviour of having owners bypass review is useful, please add a Herald rule to assign the package Owners as auditors, as currently there is no easy method to get the old behaviour back without adding individual Herald rules for every owned package.