We've had clear bug reporting guidelines in Contributing Bug Reports for a long time now, but still receive many reports which are not successful in following these guidelines. For example, in the last couple of weeks:

• T12130: Not up to date, not upstream code, no reproduction steps.
• T12122: Not up to date, resolved by update.
• T12114: Not up to date, no real reproduction steps.
• T12111: Not up to date, no real reproduction steps.
• T12095: Forked, no working reproduction steps.
• T12108: Not upstream code?
• T12102: No version information, no real reproduction steps.
• T12096: Not a bug report.
• T12083: No working reproduction steps.

We also receive a fair number of reports which sort of make some kind of effort to follow these guidelines, but have complicated reproduction cases which are not actually reproducible, like these:

• T12129: Vague, complicated reproduction steps, not reproducible.
• T12112: Complicated reproduction steps, fails to reproduce.
• T12113: Hugely complicated reproduction steps, failed to reproduce, ultimately resolved (user had already registered another account with the same email address).
• T12081: "Reproduction steps" omitted critical information where Phabricator had been manually modified, resolved in ~6 minutes with a reproduction environment.

We also receive many reports which successfully follow the guidelines. In almost all cases, the outcome of these reports is good:

• T12118: Contextualized in about 3 hours (submitted at 4AM).
• T12117: Fixed in about 3 hours (submitted at 3AM).
• T12097: Diagnosed in 15 minutes, fixed in 30 minutes.
• T12092: Fixed in 9 minutes.
• T12090: Workaround in 4 minutes, fix in 7 minutes.
• T12087: Workaround in 40 minutes, documentation fix within 24 hours.
• T12086: Fixed in 6 minutes.
• T12082: Fixed within 24 hours.
• T12080: Fixed in about 12 hours (submitted at 9:30 PM).

Finally, across all reports, things occasionally turn into long discussions (like T11597 and T9640) where everyone ends up pretty grumpy because we don't value the same things that users do and particularly because we don't assign the same value they do to our time, their time, or the value of contributions.

---

Of these, the "build a machine" reproduction cases are most concerning to me because they're very time consuming. The "discuss values" and "reporter didn't follow the rules" cases are somewhat concerning to me because they're emotionally negative for us and for users. In particular, it's personally frustrating to wake up to, say, 3 reports where no attempt seems to have been made to follow the rules and then maybe one "go build a machine from scratch to try to reproduce this crazy issue which almost certainly will not reproduce", since my next hour or two are then shot having a series of generally negative interactions that waste my time.

---

One issue with this approach is that it appears to reward users who do not follow the rules. For example, there is an obvious incentive to ignore the rules or not bother to read them when submitting a report, since that lets you submit the report faster. I think your outcome is actually much worse on average (reports that follow the rules are almost always resolved quickly; following the rules frequently resolves issues without requiring a report; reports that do not follow the rules rarely have positive outcomes), but you get something submitted faster and it isn't obvious that you're hurting yourself.

---

Tentatively, I'd like to experiment with a system like this:

Two Queues: Two Nuance support queues, basically "free" and "paid". Anyone can submit to the free queue, and support requests from free instances go to that queue too. You can submit to the paid queue by having a Phacility SAAS instance or buying a support plan.

Free queue is public, with no guarantee of a response.

Paid queue is private, with guaranteed response times (not super rigorous ones).

Community Review: Items in the free queue are initially private, and must undergo community review. In this phase, a community member confirms that the report follows the rules (for example: up to date, not forked, has valid reproduction instructions which work for them). This is a human intelligence task, but not one that requires in-depth knowledge of Phabricator. If the report follows the rules (and isn't printer fax spam), they pass it through. We then deal with it normally, in most cases by promoting it to become a real task and then moving forward normally.

While a report is in the review phase, these rules are explained to the submitter and they're offered ways to review other reports or fast-track their request by buying a support plan. (And maybe let them jump to the top of the queue by performing review?)

This is promising to me because I think it makes the cost of not following the rules more obvious, reframes reproduction instructions as stuff the community has to be able to follow, gives the user a reality check like "oh, man, I don't want to go repro any of this garbage, so probably no one will want to repro mine, maybe I should put in a bit more work" and gives users who want to contribute a way to contribute that is unambiguously valuable (vs submitting patches). (Ideally we'd just pay users who do community review, but not sure how practical that is given all the tax/legal stuff -- we could cross that bridge when we come to it.)

Critically, it gives us a free pass to ignore (or never even look at) reports which don't follow the rules.

We could also experiment with cheap promotion ("first queue promotion for $5"); I'm not sure if this would be positive or negative overall. I suspect some of the worst offenders, like T12112, place so little value on our time that they would not pay $5 even for immediate review. However, this could also create users who feel entitled to a comprehensive response because they're paying for it. We could also promote users who submit quality reports so they can skip community review, even in the free queue (or give them credits in the paid queue or something so that we don't have to have an uncomfortable conversation about not abusing your skip-review powers).

I'm not exactly sure what we'd do with general support requests. Maybe we just write up some simple rules for them too and submit them through the same process, then turn them into documentation or blog posts or Ponder questions if they're good enough. Maybe you just can't ask them in the free queue. I'd like to basically nuke 95% of Ponder and turn off open access to it.

---

Broadly, users don't seem to care too much about getting no response, or a very slow response, or even an inaccurate response. Many, many questions in Ponder have no answer or an inaccurate answer, but we don't see users complaining about that that I can recall.

Users do seem care -- sometimes, a lot -- when a human says "no" immediately.

So some of what this approach would do is make those "no" responses much slower (e.g., automatic expiry after 30 days without community review or something for the "build a box from scratch" stuff that no one has time for) and less human (still technically a human, but a human following a checklist).

---

Another thing we can do is publish average response/resolution times for queues. I think our resolution times are exemplary, but you currently only know that after we resolve something. This also makes a soft response guarantee more palatable: "guaranteed next-business-day response + median response time 4 minutes" feels a lot better than not knowing the median response time.