Page MenuHomePhabricator

Hide answer box if you asked the question in Ponder
ClosedPublic

Authored by chad on Aug 21 2015, 7:22 PM.

Details

Summary

Fixes T9241. Users have a tendancy to assume Ponder is a "forum", make replying to your own question take an additional click.

Test Plan

View my own question, see notice, click open answer box, reply. Visit not my question, see box as normal.

Diff Detail

Repository
rP Phabricator
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

chad updated this revision to Diff 33687.Aug 21 2015, 7:22 PM
chad retitled this revision from to Hide answer box if you asked the question in Ponder.
chad updated this object.
chad edited the test plan for this revision. (Show Details)
chad added a reviewer: epriestley.
chad edited the test plan for this revision. (Show Details)Aug 21 2015, 7:23 PM
epriestley accepted this revision.Aug 21 2015, 9:47 PM
epriestley edited edge metadata.

Users are pretty much all using Ponder like a forum: they're posting answers which are really comments, and asking questions which are really thread topics (very few even have a question mark in them). Maybe it should just be a forum?

This revision is now accepted and ready to land.Aug 21 2015, 9:47 PM
chad added a comment.Aug 21 2015, 9:56 PM

It doesn't seem to dis-similar to Stack Overflow, which has common quality "questions". I think just a few design tweaks should help encourage a Q&A format better. I think all Q&A platforms have this problem in general, at least it's been my experiences.

Mostly, I'm trying to keep Ponder somewhat differentiated from a "Phorum" by discouraging discussion and more emphasis on resolution.

Maybe it's futile, but there seem like some obvious low hanging design fruit.

Users are pretty much all using Ponder like a forum: they're posting answers which are really comments, and asking questions which are really thread topics (very few even have a question mark in them). Maybe it should just be a forum?

I noticed this too - I'm really excited for Ponder and would hope to have it in its current form instead of being discussion forums. I think the reason I prefer is that I could see the highlight/focus of Question and Answer being very valuable for our support team -- though if usage ends up playing out like discussion the format may not be as valuable.

With only 1 answer being allowed per user that might cause regular users to be accustomed to Comments vs. Answer. Maybe to assist in maintaining that if there was a way to demote Answers to Comments for keeping the content organized.

chad added a comment.Aug 21 2015, 9:59 PM

Threading is also a new pattern, so maybe I need to expose that a little better.

This revision was automatically updated to reflect the committed changes.

I do think some of this is just non-native speakers -- although I fixed some of them, a significant portion of the questions on this install had the form "How to <x>" (e.g., "How to remove all dashboards?"), which is a construct that I think sounds very unnatural to native speakers but which seems to be common among non-native speakers.

I'm not sure what percentage of our users are native English speakers, but I'd guess it's much higher than the percentage of native English users asking for help. I'm not sure what we could do about this -- translating the documentation seems like a pretty big undertaking, but I could understand that something like Ponder is way more attractive for a non-native speaker who isn't quite fluent than reading through huge swaths of of English-language documentation.

I had started re-wording a Question that was phrased by someone who is non-native English speaker, but reverted to just correcting "Auidt" to "Audit". I wasn't sure what the policy would be for modifying the Question's content.

I've been pretty heavy-handed about grammar corrections but I'm not really sure what the right policy is myself.

In T6920#90872, I called out a few examples of what I imagine "good questions" are for Ponder:

open ended, with no "right" answer; experience, anecdotes and context are relevant; answerer is relevant; answers are unlikely to change much over time.

I'm not seeing anything in this vein right now, though, and it sounds like you guys both think of the tool as more useful for getting a "right" answer or concrete resolution.

What sort of questions do you guys imagine being "good" for Ponder?

chad added a comment.Aug 21 2015, 10:49 PM

Anything that isn't a bug or feature request is "good" to me. The quality in IRC certainly isn't "good" but I'm still interested in hosting a place for these to exist and for community members to help each other resolve this (with proper karma-like incentives).

https://secure.phabricator.com/chatlog/channel/6/?at=211211
https://secure.phabricator.com/chatlog/channel/6/?at=211200
https://secure.phabricator.com/chatlog/channel/6/?at=211206
https://secure.phabricator.com/chatlog/channel/6/?at=211252
https://secure.phabricator.com/chatlog/channel/6/?at=211329
https://secure.phabricator.com/chatlog/channel/6/?at=211298

Maybe these are all low quality and covered in documentation, but I don't mind them.

chad added a comment.Aug 21 2015, 10:50 PM

I haven't tossed out question voting data, so if we want something for users to award good questions, it's easy to re-apply.

Ah, okay. Is your thinking that the major benefit of Ponder over a forum just the ability to induce community members to provide support by giving them some kind of karma points?

If low-quality questions (e.g., googleable, one-word answers, duplicates, etc) are perfectly fine, it probably doesn't make sense to spend time fixing grammar.

chad added a comment.Aug 22 2015, 12:01 AM

Yeah, overall I think Ponder is more about getting "help" than "deep discussion". It may be helpful to have a deeper discussion tool down the road, but a lot of Ponder is "get in, get out". I've had several community members this week message me privately about helping out with Ponder, so building a tool thats fun and centered on answering is important to me. I also think a tool like Ponder is good for strengthening a community. That is, there are means for people to contribute beyond just adding thoughts. Ponder provides a structure of "problem/solution" that people can spend 1 minute on and feel accomplished. I never really feel accomplished when using a discussion forum.

I think @chad expressed my thoughts better than I could. I don't think I could speak for the cases it would be used here however I would be happy to provide examples of questions that would be posed in my support cases. Though this is a good summation:

Anything that isn't a bug or feature request is "good" to me.

Most of ours will be along the lines of "How do I do X?", "Why did it do Y?", or "What to do if you see error message Z".

a lot of Ponder is "get in, get out"

Thinking about this a bit more, I'm a bit worried that this may lower the quality of the community in the long run.

I think the major value of the community is in attracting capable experts who have great experiences with the tool, and who drive its adoption when they switch jobs (and they sometimes they contribute directly, too). They may not personally have a high level of technical skill (and some of them aren't in technical roles), but they often do. They may not be very experienced, but many of them are.

These users are thoughtful about their actions and spend time trying to phrase their requests carefully and provide context so it's easy to help them. They file reports and ask questions that look like the reports and questions we would ask. They generally know how to file a proper bug report, generate a reproduction case, suggest a patch, operate a server, google for stuff, and so on. They also have the technical and leadership skills to effectively advocate Phabricator as a tool when they change positions or are asked for advice. They often have domain expertise in their field of endeavor, and the social and cultural awareness to be effective advocates and pleasant to interact with. They seek to understand why problems occur, and are interested in deep discussion and long-term solutions. Their reports and requests look like the reports and requests we would file, and interacting with them feels like interacting with peers.

On the other hand, some users have nothing to contribute: they bring little or nothing to the table, have no desire to improve the software, don't read the documentation, and aren't thoughtful in their actions. They don't google or make any effort to make helping them easier. They don't have expertise or technical or leadership skills and will never be effective advocates or contributors. They often aren't pleasant to interact with and helping them doesn't feel rewarding. They seek to get a quick fix and don't care about understanding anything or finding long-term solutions. The interaction generally doesn't feel like helping a peer solve a problem.

I think Quora and StackOverflow have pretty bad communities. I think Quora used to have a good community, and I enjoyed Quora a lot in ~2011, but afterward it encouraged a lot of low-effort questions and garbage, attention-seeking answers. I've disengaged from this community because I repeatedly had negative interactions with it (e.g., I was frequently asked to answer questions that were complete wastes of time, and saw lots of answers which were irrelevant-personal-anecdote garbage). I gave up on monitoring StackOverflow for Phabricator questions a long time ago because the quality of questions there is so low and it never felt rewarding.

I think stuff like Q100, where a random public user asks a really low-effort question and gets an excellent, in-depth answer from a community member is actually harmful to the health of the project in the long run. I think these requests are basically bad, and that draining the community with them is only slightly better than draining the upstream directly. Even if @cspeckmim found this interaction rewarding in some way, I think it will drive away other users like him and me in the long run.

The users I want to attract are basically peers: they have real jobs doing real work and their time is valuable. I think few will have much interest in getting internet points. To pick on Quora again, I certainly never took Quora points seriously, and I don't know anyone who did (everyone I ever saw mention them was joking about them). I think they will, on the balance, not participate in (or actively leave) a community that accepts and rewards low-effort contributions.

Put another way, I think our goal should NOT be to help as many users as possible: many users are net negatives, and the behaviors which make a user a drain on the community also often tend to make their voices louder. Users who are too lazy to google tend to be too lazy to read the guidelines, too. I think we should quite possibly be seeking to kick these users out of the community completely (or, really, just put barriers up which make it much harder for them to enter it), and that it is correct and beneficial in the long run for them to find no help or encouragement here.

In particular, I think these users are probably <1% of our userbase (and maybe <0.1%) they're just 100x louder than other users because they don't have the awareness or competence to figure things out for themselves, google, or phrase effective questions. If we had a magical barrier which rejected them we'd be kicking out a tiny number of users, but cutting down on the noise and cost generated by the community by a huge amount. I'm not completely certain this guess about how common they are is accurate, but I don't think the median (or even 5th percentile) Phabricator user can't google things.

Certainly, we know we have a large number of users at bay area companies who we basically never hear from: I think this is what the "normal" user looks like. They are competent and skillful and need very little help, but they are also busy so they don't have time to contribute or interact with the upstream. I think almost all of our users are like this, we just never actually interact with them.

So I worry that building a "get in, get out" tool caters to this tiny fraction of our userbase who are bothersome and incompetent and net drains on the community and product -- the loudest, least capable 1% of users -- and encourages and rewards their participation. I'd much rather not have these users be part of the community -- they can just go to StackOverflow anyway, which is completely friendly to this kind of stuff.

StackOverflow feels like a slum to me. I can find very little content there (either questions or answers) which reads like it was written by a peer or someone I'd want to hire or interact with daily. (This isn't to say that there's no value to it, it just isn't a community I want to be a part of or contribute to.) Ponder feels a lot like StackOverflow to me. I think it might be better if problem users find no support from the upstream or community at all than if we annex a slum and try to push them into it, at the cost of draining the users who we want to keep around.

If I think about this as a community selection problem rather than a support problem, the best technical/product course of action is probably:

  • No public write access to the community (Maniphest, Differential, Ponder, etc).
  • Formalize "Community Guidelines" which basically says: this is a privilege, not a right, and the cost of access to this resource is that you have to behave like someone everyone else wants to interact with, not a loud/obnoxious drain on everyone else. If you can't do this, go to StackOverflow.
  • The only public writable inbound channel is a Nuance queue, which is not publicly readable.
  • "Community" whitelist of users who have write access to Maniphest, Differential, etc., and probably to the new user queue itself.
  • You get added to the whitelist if you submit something that's basically positive (good bug report, good feature request, etc -- e.g., the kind of thing we'd award a badge for). If you submit something junky, we canned-response you with a pointer to the appropriate guidelines document (how to bug / how to feature / how to community).
chad added a comment.Aug 22 2015, 5:36 PM

I was mostly working under this assumption from T9212:

General support requests move to Ponder, which we mostly ignore.

Which is to build a general tool for questions and answers. You didn't overall seem happy about this though option in T9212, and with additional thought seems like you don't feel we should offer any low-barrier support options (even if these cost to us is nothing)? Is this mostly a quality issue?

There are probably two factors at play here. The first is that I just want to build a decent application with Ponder, which does 80% of whatever Quora/Stack Overflow/Yahoo Answers do. The second question is if we'd continue to keep Ponder on and point to it, which is part of T9212. Is that your overall feeling?

Is this mostly a quality issue?

Yes. I think when someone new comes to the upstream and sees a StackOverflow-level of quality, it signals that the product and community are low-quality. Even if this isn't directly draining upstream resources, I think it hurts us in the long run by encouraging users who we don't want and driving users who we do want away.

The first is that I just want to build a decent application with Ponder, which does 80% of whatever Quora/Stack Overflow/Yahoo Answers do.

I don't think we should do this. I think these communities are really low-quality communities, and a tool that emulates them will encourage a similar community. I don't think this model works for building high-quality communities.

I also think such a tool is useful for only much use for a tiny number of installs (basically: large, open installs). I think pushing gamification features might help us a bit, but probably doesn't help startups and normal companies very much, but these are who most of our user are, and who we should be building tools for. I'm not sure it even helps us, because I think the sophisticated, expert users who we'd like to be part of the community don't care about that stuff.

There are probably two factors at play here.

Yeah. I think one question is "What's the best way for "good" users (users who we want in the community) to get help with non-bug/non-feature questions?". I think Ponder might be a part of the solution for that, but I don't think these users are likely to be very motivated by internet points, or have all that many "get in, get out" sort of questions.

The other question is more like "Who should we encourage to participate in the community, and what do we want the community experience to be like?". I think Ponder in its current form encourages a lower-quality community than we currently have: new users will come and see a bunch of poorly phrased, poorly researched StackOverflow-level-of-quality content and just provide more of the same. Expert users who we actually want will see that stuff and just leave.

Basically, my personal reaction to seeing a "Q&A" tool where the bar is so low that questions aren't even phrased as question is "Oh, this is a slum like StackOverflow, I don't want to be here or be part of this". I think other users like me probably have a similar reaction.

chad added a comment.Aug 22 2015, 7:11 PM

My expectation with Ponder Moderators was that questions would be groomed. I've edited and categorized questions, and I know others have as well. This is more like Quora, but it sounds like even that bar isn't high enough here. The point of moderation features was to (hopefully) improve the quality. Certainly these could also sit in Nuance, get groomed and sent to Ponder or outright deleted.

Overall I guess my concern is basically, a user wants to use Phabricator, and is having (level requirement) trouble doing so. There are probably other core issues here, whether the documentation is too dense, difficult to search, not in their primary language, or that the UI doesn't make the answer obvious. At any rate, I'm at least in a small way sympathetic to try and help if I have the time/knowledge. I think there are other community members here that maybe feel the same way.

Maybe this also ties into a larger question of if we'd provide certain community members with (limited) access to Nuance.

My thinking has definitely changed a bit since T9212, primarily in response to seeing Ponder actually get used a bit. Here's my view on the activity that we've seen:

  • The answer to Q98 is hit #1 when searching this install for the error message ("Error while executing Task").
  • The answer to Q99 is hit #5 when googling for "phabricator staging area", and the other hits are helpful in pointing toward the answer.
  • Q100 is a low effort question + a thoughtful, high-effort answer by a community member. It looks like the answer was mostly composed by doing all the legwork to just google the question for the asker. I think this is a drain on the community in the long run.
  • Q101 basically got well-meaning but more-or-less incorrect answers but is now marked as resolved.
  • The answer to Q102 is hit #1 when Googling for "phabricator specify auditors".

Obviously this is a pretty limited sample set, but seeing these interactions occur just makes me feel like the quality bar has sunk very very far.

Overall I guess my concern is basically, a user wants to use Phabricator, and is having (level requirement) trouble doing so.

I think these users -- capable, thoughtful users who just aren't quite able to figure something out because they're missing a little operational expertise -- basically don't exist. I think these users are generally resourceful and can figure stuff out.

I don't think we actually see requests which are consistent with this kind of users.

For example, if the documentation is too dense or they're having trouble with it because of a language barrier, I'd expect to see questions like "I found document X, but I still don't understand the answer to my question after reading it", but haven't really seen this.

If search doesn't work well, I'd expect to see "I searched for X, and found Y, but it doesn't seem to answer my question". I'd also expect conducting a search for the question terms to not hit a document which provides the answer, but my experience is that it often does (as in many cases above).

Instead, I think we see a lot of requests which are consistent with users who haven't tried to read or understand the documentation, and haven't tried to google or search for answers themselves.

(I would be more sympathetic toward these users if Phabricator was more consumer-focused software like a fun adventure game for children, but it's very technical software.)

chad added a comment.Aug 22 2015, 8:00 PM

I saw Q98 and Q99 as people "testing" Ponder, per T9212, and not a specific need for support, but I may be wrong.

My plan was roughly:

  • See what comes into Ponder
  • Fix what is possible to fix
  • Decide to keep it or remove it

Is your stance basically that quality is unsurmountable? I would want a larger sample set, but if it's pointless already, it's pointless.

User inability to "search first" is a sadly a common internet problem, I don't think there is any real product solution here to prevent these questions without a moderation queue or building a "similar questions" pre-submit step.

chad added a comment.Aug 22 2015, 8:28 PM

Mostly it's not clear to me if you think we should spend any more time on Ponder. I'm fine fixing a few more things and just putting it on a shelf. I'm also fine trying to find solutions to improve the quality bar.

My plan was roughly:

That plan seems fine to me, I think I just feel a little more urgently about making sure we're on a good path.

Mostly it's not clear to me if you think we should spend any more time on Ponder.

I'm not happy with the current state of things -- I think the last 5 questions are low quality, and if the next 20 or 50 questions or whatever are similar I think we're hurting the community in the long run to stick with things as they are right now. So I don't want to shelve it: I think it's an active problem and we should either fix it or throw it away in the relatively short term.

Is your stance basically that quality is unsurmountable?

No, but I do think it will be very challenging to raise the quality bar by building a StackOverflow-like application with public access, since I think part of the quality problem on sites like StackOverflow and Yahoo Answers is that the tools tend, on the balance, to encourage low-quality questions and low-quality answers.

In particular, I think pushing gamification might decrease the quality, since I think it both encourages low-quality answers and discourages high-quality participants from engaging in the community.

A secondary consideration is that, depending on what changes we make, we might end up with a tool that solves a very niche problem (basically: public community support queues for active open source projects) but isn't very useful for most installs/users (where users are employees). I think it's a little better if we can solve the problem we face while keeping Ponder useful for most installs, although I think solving the upstream problem is more important than making Ponder do something generally useful.

For example, I could imagine a version of Ponder like this:

  • "Answer" is a field on "Question", that usually starts off blank and which everyone edits together.
  • Comments are normal comments.
  • No separate "Answer" objects. No voting. No gamification.

This version of Ponder would basically just be sort of like structured wiki pages, but might be better for @cspeckmim's use case (roughly, having employees build up a knowledge base) and the use case described in T6920#90890. It would probably not be great for the "public community support" use case that we face, but might be a more valuable tool on the balance across all installs and use cases.

I think there are a lot of ways forward here, we're just a little better off if we can find one that doesn't turn Ponder into a tool which is only useful for us. But if that's the easiest approach or the best approach we can come up with, I think that's also fine.


The two "quality" problems I'm concerned about are basically:

  1. I don't want new users to have an experience with an official upstream resource that looks and feels like a StackOverflow slum. Right now, the 5 most recent questions in Ponder communicate "this is StackOverflow and the quality bar is zero".
  2. I don't want community members to have their time wasted helping people google for stuff: these requests suck, and I want to shield the community from them as much as we shield the upstream from them.

In both cases, I'd rather users who aren't googling go ask their questions on StackOverflow, and users who like helping others google go answer on StackOverflow for StackOverflow points, and the upstream get out of the googling-for-people game and encourage thoughtful discussion and a long-term focus.

(I objected to listing StackOverflow in the "Support Resources" document, but this was primarily just a language/expectations concern: I'm fine with linking it, I just want to set expectations about it better.)


Maybe these problems aren't really problems and the first 5 questions aren't indicative of what we'll actually see in the long term, in which case we're already fine.

We could do stuff like just delete bad questions to help with (1), but this doesn't help as much with (2) -- and I'd pretty much delete everything in Ponder today if we headed down this route. If we delete the majority of questions, making the workflow private by default seems to make more sense, and I would delete the overwhelming majority of questions today.

We can put a lot more help on the create workflow ("how to ask a good question", "examples of good questions", "similar questions", etc). This will probably help a bit, but I'm not sure how much it will move the quality bar. I think other public Q&A sites have this stuff and still have major quality problems, and I think the problems are mostly caused by users who don't read or google anyway.

We could add gamification features, but I think this tends to just encourage answers and doesn't help (and may even hurt) quality, and I don't think "not enough answers" is a problem we're facing right now. Basically, I see these as primarily addressing a different set of problems (encouraging participation), not the set of problems I'm most concerned about (quality bar).

chad added a comment.Aug 22 2015, 9:43 PM

For example, I could imagine a version of Ponder like this:
"Answer" is a field on "Question", that usually starts off blank and which everyone edits together.
Comments are normal comments.
No separate "Answer" objects. No voting. No gamification.
This version of Ponder would basically just be sort of like structured wiki pages, but might be better for @cspeckmim's use case (roughly, having employees build up a knowledge base) and the use case described in T6920#90890. It would probably not be great for the "public community support" use case that we face, but >might be a more valuable tool on the balance across all installs and use cases.

That's what I suggested way back in January in T6920#90853

Oh! I totally forgot about that.

I guess I'm not really sure what Ponder should be, and I've definitely thrashed all over the place on direction for it. I think I can see a few use cases:

  • "Mini-wikis", per your suggestion: this is probably the best tool for most of the use cases we've seen in the wild.
  • "Storytelling", which is sort of what I was originally interested in in that task: open-ended questions with no right answer where the question is more like a writing prompt and the answers are somewhat narrative. This is how I used Quora in ~2011, and also roughly how Everything2 (a niche community I was part of many years ago) works. This is a use case I find personally interesting, but I'm increasingly unsure it has any place in the product. In this use case, the value of the answer is as much from reading a piece of writing as from getting the answer to a question.
  • "StackOverflow", which is what we're doing now. I'm so-so about this use case and more negative about it after seeing the initial uses. If we do make it work, I think it's more likely to be useful for us than for other installs.
chad added a comment.Aug 22 2015, 10:30 PM

Yeah, I'd agree this is kind of nebulous. I'm hesitant to push it into a "support product" that only serves large installs. Mostly, if it's not going to serve anything for us, I don't want to put much more time into it (as Nuance would be better served for us anyways).

I can add an "answer wiki" at the top and let installs just use it for general engineering questions / intern support.

Yeah, I guess let's maybe do this?

  • Just wait and see what the next ~20 questions look like, since my concern may be completely unwarranted. We can fix up the support docs in the meantime, which might help a bit too.
  • I'll see if I can prototype Nuance to a point where we can at least do really basic stuff with it, even if some of that is hard-coded for now. I think this is almost certainly the right direction for Phacility support and want to get it into production there before the end of the year anyway, and having something concrete might provide a great deal of clarity here. I think this is pretty likely to be a good direction for bug/feature request stuff, too, even if it isn't ultimately the right fit for general questions.
  • In the meantime for Ponder, let's maybe not do anything that makes it hard to switch the product to something more like "Mini-Wikis" later if Nuance ends up being a big part of the upstream solution here? Stuff like adding an answer wiki sounds great, since that seems good whether we end up in a more "StackOverflow" or "Mini-Wiki" direction, and should help us figure out if "Mini-Wiki" is good on its own.
  • Let's abandon the "Storytelling" use case, which I think is a phantom use case that only I have. We could implement status updates or add some personal blogs stuff to Phame or something if I really get a hankering to support this.

This is a very interesting discussion. I thought I would clarify my use case, though it sounds like you've discerned most of it already. My case for using Ponder in-house will not be open/public and only used in-house unlike your situation. As @epriestley mentioned it would serve more as a Knowledge Base.

We actually already have a KB and while it does have a fair bit of articles it is often not the first resource looked at by our support team. One of my lines of reasoning around this is due to the KB articles possible having so much content which is not structured in a format for relating to a problem that support would rather contact an engineer to get a brief answer. I wanted to try out Ponder as means of presenting the support team with similar situations via Question and having the Answers be brief description followed by link to KB with full details. Since the users would only be in-house support we would be able to manage the quality of Q/A much as we already do for tasks, etc. Firstly through training how to create good Q/A followed by engineering moderation when needed.

As an aside, I also have an interest in exploring Ponder as a use for the engineers posting developer Q/A around the products and codebase. I'm not entirely sure how this would flesh out yet, however I believe your descriptions of a collaborative/living Answer/KB would also apply here.

  • "Storytelling", which is sort of what I was originally interested in in that task: open-ended questions with no right answer where the question is more like a writing prompt and the answers are somewhat narrative. This is how I used Quora in ~2011, and also roughly how Everything2 (a niche community I was part of many years ago) works. This is a use case I find personally interesting, but I'm increasingly unsure it has any place in the product. In this use case, the value of the answer is as much from reading a piece of writing as from getting the answer to a question.

As a corporate user this is by far the most interesting case. Cohesive engineering practices and shared understanding of culture are terribly difficult problems. We have struggled for years with an "oral" culture. We have tried to attack that from the obvious angle of "documentation!" with mixed results:

  • Writing documentation that reflects the web of interconnections within an organization is hard. Does anyone even see enough of the web to start?
  • Updating documentation is hard, especially documentation that is meant to reflect time and place. Is this document right but too 2014-ish? Does it reflect who we are now? What does that even mean?
  • Is there actually enough consensus on controversial topics to write something that looks like documentation?
  • If somehow we got all of that right, the bar for new-hires to contribute is now absurdly high. We have made the "how do I name/categorize my new wiki page" problem far far worse.

But people like telling (or at least reading) stories --and stories don't need to "be" organized anywhere to be good, the muse just happens to reveal them -- so maybe somehow encouraging good storytelling is the right approach? For example, we would really like people to not push broken code that breaks the build. We have tried a number of approaches to that problem:

  • Say it is really important to not break the build in a wiki doc.
  • Project a list of broken builds on a big board for shaming.
  • Tell team leads that they should re-enforce that this is important.
  • Adopt this cool code review tool we found on the Internet to help with some of the technical aspects leading to broken builds.
  • In progress: Integrate pre-commit testing and review.

Now none of those are necessarily bad approaches. But they all tackle it from the perspective that broken builds are obviously bad. If you only intellectually agree that broken builds are bad but don't feel it in your bones they probably won't make a convert out of your.

A number of ex-Sun engineers I have meant talk fervently about maintaining First Customer Ship Quality at all times. Sadly there are few public tellings of that tail beyond the pale shadow of a wiki page but the impression I have gotten is (1) The crisis that lead into their version of "do not break the build" was a career defining event for many of the engineers (2) The story was told and retold so it was seared into the souls of the next generation of engineers. Probably that means they don't break the build anymore, but it is even more important if they come away with a visceral feeling for the product, not any one project, is what matters ormistakes will happen; negligence cannot.

I'm not sure exactly what our equivalents would be for the Sun story, or the whimsical examples in T6920#90856. But I'd like to find out and I think their is something uniqeuly Q&A-ish about them (where the voice of the author matters) that doesn't fit well in wiki/tickets/support-like tools. I think this (aspirationaly perhaps) fits with the long term vision for Phabricator. I view Phabricator as the tool that makes us one engineering team and not n teams, and let's us grow without fracturing.

A number of ex-Sun engineers I have meant talk fervently about maintaining First Customer Ship Quality at all times. Sadly there are few public tellings of that tail beyond the pale shadow of a wiki page but the impression I have gotten is (1) The crisis that lead into their version of "do not break the build" was a career defining event for many of the engineers (2) The story was told and retold so it was seared into the souls of the next generation of engineers. Probably that means they don't break the build anymore, but it is even more important if they come away with a visceral feeling for the product, not any one project, is what matters ormistakes will happen; negligence cannot.

If anyone happens to have taken interest in this story specifically, a few related relics were recovered recenlty. It even starts with 'Once, long ago'.

If anyone happens to have taken interest in this story specifically, a few related relics were recovered recenlty. It even starts with 'Once, long ago'.

Thanks for sharing this, I found it interesting (although I sympathize with the engineer much more than the author does, I think).

I'm happy to see more interest in storytelling, since it's a use case I also personally find compelling -- but I'm not really sure how it would best fit into the product.

It could fit in Q&A in Ponder, but I think it's at odds with the other use cases (like "get a quick, correct answer to your simple question"), and sites like StackOverflow and Quora shape user expectations about the Q&A format. It could fit in blogs in Phame, but this feels heavyweight and noninteractive, and blogs in general shape a lot of expectations. We previously had a "status update" tool (the "High Horse Soapbox") where it could fit too, but it's necessarily pretty Twitter-ey. It could fit in wikis in Phriction, but I think authorship is important in storytelling. It could possibly fit in some future discussion/forum tool. It feels a little to weird to silo in its own tool.

Broadly, Ponder/Q&A feels sort of like a local minimum in the greater application problem space to me right now, while other applications are local maxima. I don't actually know what set of problems Ponder is the best solution to.

The other tools generally have some reasonable set of use cases that they do a great job with, and then do an OK job with a lot of related use cases, and then a little further out things blur more.

Ponder doesn't feel great at anything, but like it's a sort of compromise of mediocrity in half-addressing a set of partially realized use cases.

For example, Ponder is not the best way to get a quick answer to a question with a correct answer: Nuance will clearly be much better that that.

But getting an answer from Nuance requires someone actually be responsible for answering your questions. So we're left with Ponder being an okay way to sometimes kind of get an answer to your question which might or might not be right and might or might not come very quickly, or find some similar question which hopefully has an applicable answer. In exchange for trading away responsiveness, quality, correctness, and even the assurance that you'll get an answer at all, no one has to be responsible for answering. This is sort of an acceptable compromise on the "free support" problem for us (excepting that it may dump a lot of new users into a sort of slum, which I think is bad for everyone), but seems like a problem that shouldn't really exist in normal companies.

Anyway, maybe this will get more clear once Nuance is in working condition. It might be able to relieve some of the "quick Q&A" use cases here and let Ponder have some more of the storytelling space.

chad added a comment.Sep 6 2015, 5:36 PM

My hope is to push Phame into more of an internal blogging/writing tool, so things like SEV post mortems, what the culinary team is making today, feedback on the latest Q&A with the CEO all have a place to live and disseminate. These are also probably not useful for small engineering teams/companies, but even if it's just for admins to say "Phab will be down Saturday for a migration" it should be useful

In D13957#155021, @chad wrote:

My hope is to push Phame into more of an internal blogging/writing tool, so things like SEV post mortems, what the culinary team is making today, feedback on the latest Q&A with the CEO all have a place to live and disseminate. These are also probably not useful for small engineering teams/companies, but even if it's just for admins to say "Phab will be down Saturday for a migration" it should be useful

This is great! And I see some veeery nice benefits to that approach... If phame blog accessible publicly from other domain would offer per-post policies depending on reader... Say if user is logged in company phabricator he can see all posts that are shown for internal as well external use, while logged out (or public) viewer sees only handfull of posts, that are enabled for public viewing. Best yet, it would be totally awesome to have phame post editor say somehting like "your post references non-public T1234, while it's publicly viewable, therefore it will render as {T1234} instead of T1234: differential.query by commitHash only searches the latest commit hashes" ;)

In that vein, Phame would solve one part of problems and give some benefits for companies that would want to maintain nice "marketing/public" blog + have devlog with devs able to say more (maybe add moderator role, like in Ponder).
Now Ponder would solve another part.
Then Nuance seems like it would solve A LOT of problems. Especially with multiple input sourcess.