wth with the remarkup

whew, only took a few tries

Thinking about the "Phacility" NUX case and unsure users really need to explore Config or Applications. Really we just want them to invite some people and pull in their repo. If new to Phabricator... learn about getting Arcanist up and maybe make a few projects. I'm mostly trying to separate things into "must do" and "useful tips". Useful tips can just go into Feed as the first post, like Quick Start guides for individual apps or following us on Twitter.

Also "Config" is super scary. It might be helpful to chop that up a bit?

/config/ is already an attempt at chopping it up, vs /config/all/ which is the un-chopped version. If you have a better way of making it less scary there's no need to chop it up in the way it's currently chopped up, though.

(And we can continue reducing the number of settings, but not very quickly.)

Some things in the side bar seem more deleoper related than configuration related. Trying to think where we could possibly move then maybe, like a developer app?

My intent with a lot of that stuff is to make administration and support (particularly in the future, once third-party applications are more common) easier, so they're targeted primarily at future-administrators who have questions like these, possibly because we've asked them for the information:

• Is third-party application "X" installed?
• Which extension is screwing up hovercards?

It could definitely move to some sort of "Administration/Extensions/Applications" UI, though, and isn't directly configuration-related.

The "Database", "Cache", and "Cluster" settings are more administration/setup/status than real configuration, too. And relatively few "Setup Issues" are really confiuguration-related, per se, although many of them are configuration-of-your-server-related. I don't know if "Configuration" and "Setup" should be different or not.

(And those UIs might also not actually be useful for future-administrators, particularly since I don't really plan to ever help users with third-party stuff, although who knows.)

hmmmmm

hmmmmm

ɯɯɯɯɯɥ

So first thing I notice when starting a new test instance... Phabricator wants to set my Timezone, so OK fine, click save, get redirected to settings. Is there any means of pre-populating that before the user first logs in from admin maybe?

We definitely should sync it, and probably at least some other stuff, like language and pronoun, which may not be relevant now but is likely to be relevant in the future.

It isn't trivial to sync for a few reasons: we normally sync only on registration, and Phacility logins don't go through registration. We don't have an unambiguous way, in the general case, to detect first login (and probably don't want to anyway, since if we add a new synchronizing preference we should sync it on next login?). We don't expose these settings over Conduit so that OAuth can get at them. All of this is tractable but it's more than 5 minutes of work.

I'll file something just to keep separate.

We know if someone has set up arcanist, correct? They ran arc install-certificate?

It's not 100% reliable but we can use that to make a pretty good guess.

I have 7 panels for new Phacility Admin, trying to think of one last action panel..... .... .... memes?!? Good idea or great idea?

I think we're on fairly firm technical footing on some specific changes, which I can probably lay the groundwork for and hand off to you to populate:

• Guides should become some kind of modular thing so the UI can list them, ask them for status, third-parties can add new guides, etc.
• The big steps immediately below "Guides" ("Quests"?) should become some kind of modular thing so that each Guide can list all of its quests, show a completion percentage/bar, third-parties (and Phacility) can add/disable quests, etc.
• We need some kind of storage to keep track of stuff you've skipped. Probably per-user? Although maybe this actually shouldn't be per-user, or should be sort of mixed. For example, if I skip "User Account Settings", it makes sense that it only skips that step for me (not for other admins/users). But if I skip "Personalize Your Install", that should probably be skipped for all users? But if we do this, it should probably show "Skipped by epriestley".

I'd like to try to have a better sense of some stuff before building this out, though, since I'm not sure I'd get some of it right. Here's stuff I'm not sure about:

Quest Details: When you click a "quest", you currently get dumped somewhere else in Phabricator, with no context and no way to get back. For some, like "Notification Server" taking you to the Notification config setting, no reasonable user who isn't already an expert at configuring Phabricator could possibly guess what they're supposed to do.

I'd like to get a better sense of what this flow will look like and how it will work. If we're just sending the user somewhere we should probably send them to the documentation (e.g., Notifications User Guide: Setup and Configuration instead of /config/notification/) but that doesn't feel great to me. For at least some steps, having quests is still technically better than just having a "Quick Start" guide in the documentation since we can at least keep track of what you've done, and automatically detect that some steps are completed. But I think we could do a lot better than this, and even if we don't build it today I'd like to have a better idea of what we'd want to build in an ideal world.

For example: do quests have individual sub-steps? I could imagine this quest taking you to a "quest detail" page like:

• A few sentences about what this is, what it does, why you want to configure it, and how hard it is to configure.
• A link to the documentation.
• Sub-steps like:
  • Configure the server (completed by setting notification.servers).
  • Some fuzzy stuff in the middle here where we try to detect server config.
  • Send a test notification (completed by delivering a real-time notification to a client).
• Learn More / Next Steps (like a link to the "Notification Server" status page).

The fuzzy stuff in the middle is a little tricky because there's some stuff we can detect and some stuff we can't really detect.

I don't know if these sub-steps would ever be checked off manually. If they are, we need additional storage for keeping track of sub-step states.

This also just feels like a ton of technical work. I could imagine spending a day or two just implementing the technical aspect of this (e.g., trying to automatically detect issues and tell the user what they need to do next).

Context: I touched on this above, but getting shipped off to somewhere else, sometimes with little or no clue what you're supposed to do -- and no way to get back -- feels pretty confusing to me. If I click the wrong thing, I can immediately be lost without any idea what I'm supposed to do next, especially for the more complex steps (like "Login and Registration" and "Notification Server").

Putting a "Quest Details" page behind the link would help, but I think we should also really look at building some kind of "Quest Tracker" which sticks around, tells you what to do, and gives you a way to bail out and go back to the guide. I don't know exactly what this would look like:

• Left-side column?
• Menu item?
• Floaty clippy friend?
• Subheader bar?

None of these feel really great to me, mostly because getting any element to work well in Conpherence and on Workboards with the durable column enabled and everywhere with the setup banner complaining and on mobile is a bit of a mess. But dumping users into Auth or Notifications Config or Diffusion with almost no actual guidance feels worse.

Teaching: Some of the existing steps (like "User Account Settings") are covering important topics, but not actually teaching the user how to do the tasks on their own. For example, the settings quest just jumps you directly to settings without showing you (or even telling you) how to get there. We don't actually explain to the user that the wrench icon takes them to settings, and don't require them to demonstrate that they've learned it. Same with "Create a Project", "Create a Repository", "Create a Dashboard", etc.

A user who completes this stuff can't necessarily do the same task again, since we haven't actually shown them how to get started. They will be a little bit familiar with the actual process if they can find their way back, at least, but probably won't know where to go to get started.

I'm not totally sure what the best way to tackle this is, but "Quest Details" / "Quest Tracker" probably help?

Quest Goals and Content: This is sort of tangential, but some of the quests aren't very focused on what they're trying to teach the user right now, or could be better ordered, or whatever else.

For example, the "Create a Project" quest is potentially sort of confusing/misleading/unfocused: users generally don't need to create projects right away, and we haven't seen confusion around how to create projects (at least, that I can recall). I also think JIRA has the same setup step, but projects in JIRA are pretty different from projects in Phabricator, so we're showing users who have installed JIRA something very familiar with an obvious parallel but it's actually covering a fairly different topic.

In this case, the primary goal I'd have for this quest is to teach users / set expectations about how Phabricator projects work and how they differ from JIRA projects. It actually sort of drives against this goal to just ship them to Projects and have them create anything without guidance, since that will probably cement the user in whatever thinking they have about projects already: if we just say "create a project" and they're familiar with JIRA, they'll probably create a project like a JIRA project, and then the quest tracker will say "great job", and they'll think "okay, I did that right so this is just like JIRA". But this is the wrong lesson, and we've reinforced a misunderstanding.

I think the "Invite Collaborators" step should be at the end -- as structured, you'll potentially invite collaborators to an empty install with no hints about what they should do next, which changes under them as they click around because the administrator is completing "Personalize your Install". And I'd guess a lot of users don't want to invite others to something they don't understand and haven't configured yet.

A lot of the content is kind of placeholdery right now, but I think the content really shapes what the application should be able to do, and it's hard for me to be confident that I'm building the right technical things if the content is placeholderey. In the "Settings" example above, we need different capabilities if we're just going to dump users into Settings (as we do today) versus show them a screenshot of what to click, versus pop up a quest tracker that tells them where to click, versus pop up a fancy quest tracker that tells and show them where to click. But I think we should be more toward the fancy end even with v1, since the content goal for this quest in my mind is primarily "teach the user how to find settings", and we fail to achieve that goal if we dump them into Settings directly without explaining how to get there.

Non-Admin Experience: I don't have a clear picture of the non-admin experience, particularly in the future. At least some of the steps ("Settings", "Create an X") could be completed by users, and some of the planned steps ("Configure Arcanist") must be completed by all users (except that some installs hate arc, some users won't do code review, and so on, so maybe this is ultimately quite complicated).

We can just mark some of these steps as user-completeable and show users the same UI as admins, just without the admin guides/quests. But I'm not sure if that's really the right way to go about things. Particularly, this might leave new users with a bunch of "Create an X" quests (which we generally don't want them to complete -- it's bad if every new user creates a repository and a project). If we just did this with the current content, the result would be pretty weird/unfocused ("Installation Guide" with one step, "Quick Start" guide about just creating a bunch of random objects).

I'd like a clearer picture of what the actual new non-admin user experience looks like. If it's "arcanist, differential, maniphest, dashboards", what about nontechnical users? What about installs that hate arc? Why is dashboards one of the four most important things we want to walk a new user through? Can a new user possibly configure a relevant dashboard before spending some time in Phabricator? How could they possibly know what to put on it? Are we really going to tell them to set up arc before they can use Phabricator? Do we even want new users to touch Dashboards today given how rough it is (T7216, T10612)?

Updates: I'd like a clearer picture of the ongoing role this UI has across Phabricator updates, at least in theory / in v2 / in the future. When we release a new application or a user installs a third-party application, does it come with a guide? How do they find the guide? How do we show them that there are new quests they can complete? If new applications come with a guide, do first-party applications have dedicated guides?

Even without going this far, suppose we unprototype an administrative capability like SMS. Do we write a guide/quest for it? How do we show that to existing administrators?

Maybe the answer here is generally "no, this is just minimal setup for first-time users". But, if so, maybe steps like "Notification Server" shouldn't even be part of this.

Testing and Complexity: I'm generally worried about how much complexity we'll be adding by building this stuff out, and how rarely it will be tested.

Even if I want to test NUX, there's currently no real or easy way for me to reset it. For some steps, we actually could build a "reset" button: for example, the "Change a Setting" quest could store the current version of your settings when you click "Reset", so the quest would become completable again after a reset. Likewise, "Create a Repository" and "Create a Project" could store the largest repository/project ID, and only "count" newly created stuff after that. But I think there's not too much we can do about config-based stuff, auth, invites, etc. And even if we do build this, it adds complexity by creating multiple "ready" conditions: either there are no objects, or there are some objects but there's also a marker that says they should be ignored. This creates the possibility that resetting the quest works fine but the actual NUX that new users hit doesn't.

Realistically, I think no one is going to test NUX regularly. It's hard to test and hard to even remember that you need to test it. This makes me wary about building features like "quest tracker that highlights the part of the UI you have to click", even though I think that would make the quest a lot better, since we probably wouldn't notice if we broke it after, e.g., changing the logo and accidentally changing some of the HTML. And breaking NUX is pretty bad, since it gives users a first impression that the software is super awful and super broken.

I think there's also pretty much no way to put automated testing on most of this. Even if we had headless browser testing like Selenium, I think it's basically impossible to write a robust, convincing test for "is the wrench icon highlighted nicely?".

I think the current version ("dump users in other parts of Phabricator with no context") is a pretty bad experience, but fairly robust and hard to break (only breaks if URIs change). (Although, the "Notification Server" URI is currently wrong -- not sure if that changed or is just a typo.)

One step up might be "Quest Details" with text descriptions or screenshots of what to click. These break a bit when we make UI changes (and we'll ~never remember to fix them) but humans can usually still figure out what's going on.

A step up from there might be a "Quest Tracker", some kind of persistent element which follows you around and just describes what you're supposed to do next.

And a step up from there is "Fancy Quest Tracker", which follows you around and highlights/focuses/animates/points-at elements you're supposed to interact with (e.g., draws your attention to the "Settings" wrench when you need to click it to continue).

Somewhere in the spectrum of "Quest Tracker" we also end up with complexity on the backend, e.g. does the text/guidance change when you make it to the "Settings" page? If it does, we need to detect how far you've gotten.

In some other cases, we can also make a quest easier to complete if we add complexity to the application. An example is the "Notification Server" quest: we could complete it when the user actually receives a real-time notification in their client. But we don't currently store this or have a way to query for it (the client never tells the server that it got a notification). We could add such a mechanism, but it would be almost invisible when you're not specifically completing the quest.

Even when we don't add new stuff, the approach some quests take today (querying into other applications in an ad-hoc way) is fragile and easy to break.

And any new global UI we add has been consistently fragile in the past, and tends to interact poorly with other global UI.

I don't really have any great answers for any of this. Adding this complexity seems like it makes the application much better, but will also make it harder to maintain and greatly increase the chance that we ship broken NUX and don't realize it. One small thing we can do is try to share any code used by NUX in other parts of the application so at least it's running more often and it's more obvious if it's completely broken, but I don't really see a way to use a "quest tracker" or a "show the user where to click" capability in any other application offhand, and many things we could do ("has the user made it to the settings page?") are almost necessarily only useful for quest tracking.

If we did use this as a more regular way to guide users through content updates and new features that would somewhat help with testing since we'd be touching the code more often, but also probably add a lot more fragile stuff to the codebase that had the potential to break.

We can do something like "I manually run through the entire NUX every week before we cut the release", but I'm not very excited about that.

Anyway, I don't have much of a pathway forward here, I guess. I'm worried that if we just take the short path to shipping this (M1467 + Guides roughly as it exists today) and figure the rest out later we're going to be shipping a pretty janky experience (lots of sending users off cliffs to their deaths by dumping them places with no idea what they should be doing), and the technical pathway isn't all that short (I have to modularize the guides and quests and build storage and interactions), and I'll probably get a lot of stuff kind of wrong and have more work to do when we come back and fix it.

And if we take a longer path, the technical side gets a lot longer (tons of complexity in writing steps/checks for walking users through stuff) and I think the application has an inherently high maintenance cost.

Maybe an alternative is to just ship the new default dashboard from M1467 and give the "Repositories" panel a simple magic NUX state (button linking to docs pretty much) and the "Feed" panel a simple magic NUX story. However, even that may have a moderately long technical pathway through Dashboards given how much open stuff there is there, and if the magic first feed story is going to mention Phacility stuff it needs to be modular.

Yeah, I think scope-wise this could get out of hand quickly and consume a month on both design and engineering. I think I can propose something here that will not take your time and maybe 1-2 days of mine.

• Hardcode a fake version of M1467, as a replacement for the current hardcoded page that doesn't look like a dashboard.
• Swap in the new guide, also hardcoded like current guide, with no tracking
• Some sort of Simple NUX for repository, which should be easy if we faked M1467?

Basically, I'm fine burning an evening or two to test out new NUX layouts and see results. Working through long-term plans can wait until we have better infrastructure (like new Dashboards). Thoughts?

Though I guess a "fake" dashboard is not as performant as actual dashboards. But equivalent to current hardcoded homepage?

Maybe it might be a good idea to allow people to create own Guides? Could be helpful for recuring setup stuff for newusers for example, like we got it at our instance.

You can today, just extend PhabricatorGuideModule and drop it in src/extensions. It should just show up.

We'll build out guides more completely over time and eventually have them user generated.