This is a followup to the Conpherence discussion about documenting WMF reporting use cases; it is intended to support Phacility's research into implementing reporting.
- As a Product Manager, I want to compare my gut feeling of amount and type of team activity over the last quarter versus reality (as measured by Phabricator task completion) so that I can get a reality check and adjust my gut feeling.
- As a Product Manager, I want to find out if a team's recent velocity is down or unplanned work is up, so that I can decide how to triage upcoming work.
- As a Team Manager, I want to see levels and trends in team velocity so that I can make better informed staffing level decisions.
Design and Implementation issues
- The fundamental issue in providing this information to users is, what is the scope of tasks to be reported on? The general answer is, all tasks that my team is responsible for and none that they aren't, but defining this in Phabricator has varied from "all tasks in project X" to combination of projects to combinations of intersections of projects and even more complicated arrangements. If a report supported only "all tasks in project X", this inflexibility would probably simplify the effort of getting different teams to use consistent processes.
- The division of velocity into subcategories is, in the sample, based on fairly complicated criteria.
- Some teams like reporting by sum of points, but most don't use points and so count is the only metric available.
- The sample report reports by rolling time periods (e.g., the last ~30.5 days, not "March to date"), for two reasons: teams like seeing feedback is quickly as possible, as a sort of "reward" for closing stories; and because it keeps all data points comparable.
- Teams have found value in weekly, bi-weekly, monthly, and quarterly summaries. Monthly is probably the single most universally useful resolution, because it is small enough to provide feedback in time for teams to use it, but large enough to hide a lot of noise.