whether upstream wants to wait until it can tackle the generalized problem, or whether you'd be okay chipping away at smaller use cases first,

I don't think they're very separable.

I expect the cost to go from 1 ruleset to 2 rulesets is 99% of the cost here, and that adding additional rulesets will be mostly trivial. We have a fairly well established pattern for modular extensibility at this point and everything we apply it to breaks down more or less like that -- getting the core is the hard part. We can start with "all must accept" only, but I suspect this won't meaningfully reduce the implementation cost (and my gut is that doing both "all must accept" and "N of M must accept" at the same time will probably result in fewer API changes later and be a net benefit overall, although perhaps the cost to select "N" for rules like "N of M" or "at least X% of M" is high enough to justify holding off on it).

The implementation will look like essentially all other modern modular extension points look:

abstract class DifferentialReviewerRuleset { ... }

final class DifferentialReviewerAnyRuleset { ... }

final class DifferentialReviewerAllRuleset { ... }

These classes will have some methods like (for "all reviewers"):

public function shouldMarkRevisionAccepted(DifferentialRevision $revision, array $reviewers) { 
  if (!$reviewers) {
    return false;
  }

  $all_accepted = true;
  foreach ($reviewers as $reviewer) {
    switch ($reviewer->getStatus()) {
      case DifferentialReviewerStatus::ACCEPTED:
      case DifferentialReviewerStatus::ACCEPTED_PREVIOUS:
        break;
      default:
        $all_accepted = false;
        break;
    }
  }

  return $all_accepted;
}

The UI and Editor classes will instantiate the ruleset and make a call to it. The block of logic in DifferentialTransactionEditor->applyFinalEffects() which implements the "any reviewer" ruleset will be replaced with, approximately:

$object->getReviewerRuleset()->shouldMarkRevisionAccepted(
  $object,
  $object->getReviewerStatus());

Particularly, this will let you implement a ruleset yourself:

final class CustomAllReviewersUnlessTheyAreASeniorEngineerInWhichCaseJustThemRuleset { ... }

Possibly, these rulesets should really be generalized across Differential + Audit, although I think "all must accept" is more clear-cut in audit.

So this part is very straightforward, and would take only a few minutes to implement. However, it would be very difficult to understand or use and would almost certainly imply a substantial support cost for us by generating a lot of questions from users. I'm tentatively fine with implementing a very small version of this class, but I'm not sure a small version exists that isn't confusing / won't create a support problem in the upstream. Most of the big problems here are support / usability problems. In particular:

• How do we decide which ruleset a revision uses? How does a user choose it? How do we show it in the UI? How do we explain what the rulesets mean to users?
• How do we show users what the requirements to get a revision to move to "Accepted" are?
• Can users change rulesets? How? What, if anything, prevents users from changing from a strict ruleset to a liberal ruleset to avoid review rules?
• The bucketing problem is its own challenge and described separately in T9372, although I think it is avoidable for the "All Reviewers" ruleset, specifically, because the bucketing rules are not different. For "M of N", the bucketing rules are different, and are not reasonably database-expressible.

Some of those aren't too hard and we can brute force our way through them to some degree (e.g., it's reasonable to put ruleset explanations behind some kind of "What does this mean?" link) but some of those questions have simple answers which would make Differential significantly more daunting for new users and/or harder to support for us. I suspect we can find much better answers to most of those questions by designing the interactions more carefully.

For example, a naive approach would be to put a "Reviewer ruleset:" field in arc diff where you enter "all" or "any", let users change it freely, put a big blob of text on the revision view explaining what's going on, add a Herald action for changing it, and dump it into the attributes on the list view. This would technically work, but it's substantially inelegant and almost certainly not the best approach we could come up with. "Reviewer Ruleset" would be confusing and meaningless for new users, the big blob of text would be pointless for experienced users, we'd get tickets about users degrading rulesets when they shouldn't sooner or later, and so on.

when in the roadmap this might fit in,

T8783, T9378 and T9379 are from you guys so we can put it ahead of those. I'll update and reconcile Prioritized soon (next few days?) to provide a clearer picture here. T9123 and T9252 are moving out of focus (we've hosted our own builds since earlier this week) but T182 has been optioned in and some other things have shuffled around.

how much work it would take to build

I don't know what the answers to most of those UI / usability questions are yet. I'd guess it's like 8 hours of work total, but only after thinking about it for a week.

This may be more involved if it triggers migrating reviewers from edges to a dedicated table, which I think should happen at some point. Doing so may simplify some of the UI questions.

Your particular issue is also possibly fixable by sidestepping the entire issue, solving T9372 instead, and then leaving "Accepted" revisions which you have not acted on in your queue rather than moving them to "Waiting on Others". I think this would be a reasonable default, particularly if bucketing was configurable so that users who wanted to ignore these revisions could reconfigure bucketing to have a similar behavior to the previous behavior.

Getting a ruleset for N of M reviewers would be enormously helpful, if while you are doing this you see a good way to add a time criteria that would help us out. Our current ruleset is roughly ( (90% of reviewers approve) AND (no one requested changes) ) OR ( (it has been 8 business hours) AND (at lease 2 reviewers approve) ) We anticipate in the future we may need to prove this rule is being followed so we are very interested in this feature.

How do we decide which ruleset a revision uses? How does a user choose it? How do we show it in the UI?
Can users change rulesets? How? What, if anything, prevents users from changing from a strict ruleset to a liberal ruleset to avoid review rules?

I imagine the ruleset would be assigned as a Herald Rule and treated much the same as add reviewers and blocking reviewers. I think adding rulesets and blocking rulesets also makes sense. The only way to change which rulesets are applied would be to change the Herald rules.

This will probably use the stuff from T9515: Build a "quorum" mechanism and associated UI.

T8887 is also related to this task

See PHI1378, which is interested in "at least N", for N = 2.