We could make better decisions about removing support for old versions of software, removing config options, and prioritizing applications if we had usage information. For example, it would greatly inform our ability to answer these questions:
- When can we drop support for PHP 5.2 (T7408)?
- Config option `x.y` seems silly -- can we just remove it, or is it being used?
- Can we throw Ponder away?
The best answers to questions like these involve balancing costs against benefits, but we often have to guess at the magnitude of the benefits right now. Although we seem to be doing OK on this overall, it does cause us to make some very conservative choices (like retaining support for old versions of things) which might not really be necessary.
Overall, I'd like to collect this data:
- Version of PHP and MySQL.
- Version of phabricator, arcanist, and libphutil.
- Version of `hg`, `git`, and `svn`.
- Which config options have been changed from the default values.
- Which setup issues are unresolved or ignored.
I think this data would be interesting, but it begins to ebb into less-anonymous territory, and I think it is less valuable for us to collect (fewer decisions would be informed by having it):
- Order of magnitude counts for major objects: users, tasks, revisions, wiki pages, etc.
From a user perspective, these UX concerns are important:
- It should be opt in.
- It should clearly explain what data is being collected, and why.
- It should show users exactly what data will be sent and let them confirm that they want to transmit it.
- It should be as non-annoying as possible, and easy to dismiss permanently.
I think the trickiest part of this is UX around "non-annoying". Maybe something like a notification which doesn't fire until an install has been up for at least a week? But then we have to delete the notification if one administrator completes the process...