I'm close to interacting with this stuff on Herald outbound rules -- do you have a general attitude toward the future of mail tags?

I personally think they're really super ultra bad, particularly this page:

https://secure.phabricator.com/settings/panel/emailpreferences/

It has ~60 options today, and would probably have ~100 if every application had support.

My general assumption is that this is totally unreasonable and an awful UI and that we should be trying to get rid of it. From an engineering perspective, it, e.g., seems real dumb that every application has a "subscribers" tag and I have to go turn off 20 of them to turn off subscribers mail.

However, users don't really seem to mind it (I don't think anyone has complained about it) and the issues they have run into that I can recall are extremely granular (e.g., mail from specific repositories) and don't really point at this UI being a problem. And I think you don't have as negative a reaction to this as I do, either?

Generally, I guess, do you imagine this having 100+ options a couple years from now and feel fairly good about that?

I turned off all subscriber mail too, if that helps provide any additional insight. I wondered the other day if this should just be "free" somehow.

I hate to suggest a basic and advanced view but... I'd rather build the easiest thing for users to understand. Like, what settings do 80% of users really want and for the other 20% there is advanced + herald.

100 options doesn't specifically bother me, I think that form is very clear and easy to operate, it's just long.

I also wonder, if we correctly solved this, if role profiles for email is even needed. I'd rather take a pass at something awesome/magical and remove the burden from admins if possible.

Even just maybe "apps I've used" might be a quick way to reduce the long list? Or... apps that have sent me mail?

One thing that I'll probably do for Herald, at least, is group some of the tags up so you can write this rule:

When:
[ Mail tags ][ contain ][ Any Application: Subscribers Changed ]
Take these actions:
[ Deliver as notification ]

Currently, you'd need to write this rule as:

When:
[ Mail tags ][ contain ][ Audit: Subscribers Changed, Maniphest: Subscribers Changed, Phriction: Subscribers Changed, ... ]
Take these actions:
[ Deliver as notification ]

...which is even worse than just having 100 options.

I could do that without changing the settings page, or we could maybe surface some defaults on the settings page at the top, so you set a default setting for "Subscribers" and then can specialize it per-application if you really want. Maybe the "basic" could be:

• Subscribers change
• Projects change
• Members/reviewers/auditors change

And then "advanced" could be all the per-application versions of that.

But I also worry that if this is really a sort of social / work habits problem (i.e., some users feel like they have "too much email" when they get 5 mails a day, some when they get 50, some when they get 500, and outbound mail volume is not generally correlated with users feeling like they receive "too much email") we may be making the problem worse by adding more and more layers of complexity.

What about a "welcome new user, go set up your profile and preferences" nux?

Honestly I'd be fine removing subscriber emails and making them Herald only.

My 'useful' bar for email is "would I want to reply to this".

I think there are a lot of reasonable things we can do to make it easier to get your settings set the way you want them, but they'll only help users who have a "oh, these settings aren't very good, let me go fix them" mindset. My concern is that the root of this problem may not be users having difficulty setting things up, it's users just feeling helpless and overwhelmed by the problem for essentially personal/emotional reasons which we can't easily fix.

When they get too much email, I think they aren't looking at the settings, trying to tweak them to work better for them, and then complaining to administrators when the settings prove too complex or they can't get the specifics right. Instead, they're just complaining right away, or complaining indirectly, or complaining later when pressed.

There are some problems here which are genuinely "this isn't powerful enough to do what I want", but these reports are from users who are of the mindset "I want to set up a filter to do X". Here are some examples:

• (Original request on T5791) When I set up a herald rule, it doesn't include the repository as a separate token, so it's not possible to do things like set up a gmail filter based on the particular repository.
• (From T6296) Request is to offer a setting that allows "Do not notify me of changes when the task's subscribers change unless I am reporter of the task", as Bugzilla offers this functionality.
• (T5769) Add a setting to not trigger notifications when moving task across workboard columns.

These are clearly "I want something specific, and there isn't enough power to do it right now", usually because the user uses Gmail and it can't filter on headers.

Compare that to the other use case, which is "my users are overwhelmed by email":

• (T9166) people have complained about the overwhelming amount on emails they receive
• (T7013) One of our most common internal complaints is 'phabricator sends too much email!!!'.
• My own oft-related tale at Facebook where I got 11x as much mail as the overwhelmed user.

I think these users are just giving up immediately. I'm not sure if this is true, but I'd expect these reports to be more like the first kind of report if they were really the same problem: "a couple of users have a hard time separating mail of types X and Y" or "several users have complained that settings page Z is too hard/confusing/time consuming".

If we treat this purely as an emotional reaction problem that is more or less uncorrelated to outbound volume, I think the solutions are different:

Trying to head users off at the pass ("Welcome! Let's get you set up...") and get them before they're frustrated might work, but it could also overwhelm them. They also probably don't have enough context to do this correctly when they first log in.

We could try to provide more objective information about mail volumes. For users who aren't near the top of outbound volume, we can tell them "You receive 8% as much mail as the most active users.". For users with few filtered mails, we can tell them "You aren't filtering very much mail.". This might make users feel like they shouldn't be as overwhelmed as they are. At least among people I've interacted with directly, I think there's a sort of cultural stance that email is bad and inherently overwhelming (cf. Asana's whole company, the idea of "Inbox Zero", etc). I've never experienced this personally, so I don't believe it to be true in the general case, but maybe some users have their ideas about email framed by this way of thinking.

We can try to add a "suicide hotline" to mail. Currently, we have a link to mail preferences labeled "EMAIL PREFERENCES". This is great if you're in a "oh, let me go fix my settings" mindset, but presumably overwhelming if you're in "I get too much email, and email is inherently overwhelming" mindset. Instead, this could be:

• Call it "I DON'T WANT THIS MAIL".
• Link to the mail/132/ page for the message.
• Have a big "I Don't want this mail" button there.
• This exposes settings in a simplified way, by selecting just those settings relevant to the specific message. For example, if the mail was from Maniphest and about a subscribers change, it might have these options:

[ Don't send me mail about subscriber changes in Maniphest ]
[ Don't send me mail about subscriber changes in any application ]
[ Don't send me mail, ever ]

It could have some links to documentation ("Help! I Get Too Much Mail!") and the full settings page.

I'd also like to better understand why I can receive 10x the amount of mail that other users do and not think twice about it while they get totally overwhelmed, but I'd guess this boils down to some human thing that we can't really fix and that probably won't inform the problem much over just thinking about these users as getting emotionally overwhelmed by receiving a very small volume of email.

We can also pare back the amount of mail we send (e.g., subscriptions, project changes), but I think it's worth considering paring it back completely, to zero. The major advantage with this is that it would make it very clear to administrators that the problem was between them and their users in all cases where the issue was emotional, and we'd only receive actionable reports (settings aren't powerful enough to do X).

It's also a lot easier to onboard administrators, who are in a motivated mindset, and tell them to turn settings on, than it is to pull overwhelmed users out of their hysteria and get them back on track.

Put another way, "send no mail by default" solves a different problem, which is "administrators complain to us that their users complain to them that they get too much email, but some of this is rooted in personal/emotional issues and has no technical/product solution".

Maybe better feedback to get from people is "what mail did you receive that was unexpected or not useful"?

Yeah. I'd also really like the objective numbers from bin/mail volume, but they won't be very good for 30 days. Maybe we just wait that long to attack the "too much email" aspect of this -- role profiles and herald outbound rules are justified on their own anyway. I'd want to get this data:

For each user complaining that they receive too much mail:

• From bin/mail volume, how much mail did they actually receive in the last 30 days? How much mail did the top few users receive?
• From bin/mail volume, what percentage of outbound mail are they filtering?
• What are some examples of mail they currently receive, but would prefer not to receive?
• What are the actions the user has taken to try to fix the problem themselves? What was the outcome of those action?
• Why isn't adjusting mail settings a good solution for this problem? For example:
  • Have they adjusted their settings?
  • Are they having trouble finding the interface?
  • Are they having trouble using the interface to select the settings they want?
• Why aren't client mail filters a good solution for this problem?

In fairness, I think self-actions should be a part of this conversation also.

(is that tracked?)

bin/mail volume doesn't report what fraction of mail is generated by the user, if that's what you mean.

I think "Self Actions" are a good example of this possibly not really being a technical problem: they're very very easy to disable, and clearly called out at the top of the settings page. Anyone who doesn't want them and is in a "oh, let's adjust settings to fix this problem" mindset should be able to turn them off very easily, in much less time than it takes to complain about email. No redesign of the settings page could plausibly make them more accessible, so if they aren't good enough in their current form I think we aren't facing a technical problem.

We do record the originator of the mail internally, and could expose it in /mail/ or bin/mail volume, etc., without much work. For example, it would be easy to add:

[ Don't send me mail about my own actions ]

...to the "Suicide Hotline" workflow.

I would like to see that tracked as well in the workflows.

I think it's also hard to find the email preferences UX wise (sans email). It's under the fold. Maybe we can set an order to the settings panels to surface email higher.

One more idea, maybe "GROUP" settings? Everything seems to fall into a groupable setting. This could be the "basic" view.

• Details: Object's details have changed
• Status: Object's status has changed
• Comment: Object has a new comment
• Subscriber: Object's subscribers have changed
• Other: Other activity occurs

Or maybe even only have these 5 settings, and everything else is Herald.

In T9161#131972, @chad wrote:

Or maybe even only have these 5 settings, and everything else is Herald.

+1 for this. I think the 60+ options form is a little daunting, and the basics can easily handle most cases.

There's a lot of back-an-forth here but @epriestley pointed one thing in particular that reasonates well with my experience:

In T9161#131955, @epriestley wrote:

I think these users are just giving up immediately. I'm not sure if this is true, but I'd expect these reports to be more like the first kind of report if they were really the same problem: "a couple of users have a hard time separating mail of types X and Y" or "several users have complained that settings page Z is too hard/confusing/time consuming".

That is very true, especially for users that: are not especially technical (sure, to use phabricaton one is almost expected to be technical, but there might be non-technical users), that do not grasp english so well etc...

Trying to head users off at the pass ("Welcome! Let's get you set up...") and get them before they're frustrated might work, but it could also overwhelm them. They also probably don't have enough context to do this correctly when they first log in.

That won't work.. People that know phabricator might appreciate being able to set-up preferences at start, but those that know phabricator would do it anyway. Without context user won't be able to make proper decision about mail.

We can try to add a "suicide hotline" to mail. Currently, we have a link to mail preferences labeled "EMAIL PREFERENCES". This is great if you're in a "oh, let me go fix my settings" mindset, but presumably overwhelming if you're in "I get too much email, and email is inherently overwhelming" mindset. Instead, this could be:

• Call it "I DON'T WANT THIS MAIL".
• Link to the mail/132/ page for the message.
• Have a big "I Don't want this mail" button there.
• This exposes settings in a simplified way, by selecting just those settings relevant to the specific message. For example, if the mail was from Maniphest and about a subscribers change, it might have these options:

[ Don't send me mail about subscriber changes in Maniphest ]
[ Don't send me mail about subscriber changes in any application ]
[ Don't send me mail, ever ]

Now this is what I was referring to! Please do this and almost all "i get too much mail" people would STFU. We did something similar, with notion that "don't send mail ever" means that we only send invoices and/or password reminders.

In T9161#131941, @chad wrote:

My 'useful' bar for email is "would I want to reply to this".

+1

At my company, the majority of non-technical users simply turned off email notifications entirely. I know it won't happen but it would be awesome to be able to say "turn off everything but subscribed Maniphest tasks for this user" because then I could still get them to reply to tasks that I've cc'd then on. Otherwise, I have to have the conversation outside of phab and keep the task updated myself.

Getting executives/non-technical users to actually take the minute to set notifications up properly is like pulling teeth.

I think these users are just giving up immediately.

This is true for both the instance I use at work, and the personal Phabricator instance I run. People see that the noise to signal ratio of Phabricator emails is about 30:1 or greater and immediately disable all emails. They don't want to pick individual email options, they want reasonable defaults. Some of the options aren't even clear either, like turning off email for the "Other" category in Audit disables any Harbormaster build failure notifications for commits. I mean, I understand why technically this is the case, but it doesn't make any sense to a user. And since developers don't know this, they go through and turn off everything in the list except the things they think they want, and this has the consequence that developers don't know that they've broken the build (until someone else verbally tells them, or messages them through Hangouts).

This extends to code reviews as well; some developers have all email turned off, so if you want them to do a code review you need to tell them. It isn't clear who has code review notifications off either, so sometimes you're waiting a few hours for a code review before you go over and find out they don't know anything about it because they haven't checked the Phabricator dashboard in a few hours.

The developers at my work already get swamped with emails; emails from product, emails from management, emails from automated systems that already have a high noise to signal to ratio and Phabricator is just something else to add to that latter category. I wouldn't be surprised if people filter the emails in Gmail rather than trying to dig into the email settings to turn them off in Phabricator either.

This basically boils down to: the defaults are not good. No-one wants to receive emails when someone else CC's them on a task or code review that you might only be tangentially related to. Do I even care that someone else has CC'd on a code review that I'm doing? Probably not unless they take some other action (like commenting).

I think a big part of what the defaults should be driven by is: Does the user have to do something about this email? For something things, like a code review being rejected or accepted, the answer is obviously yes. For things like CC's or revisions being closed, probably not? Some of these actions differ in their importance based on whether the user had any interaction with the object. If I accepted a revision, do I care if it's closed? Yes. If I was just added as a reviewer and I haven't even opened the revision and someone else has accepted it in the mean time, do I care? Probably not (and in any case, I have the email of being added as a reviewer still in my inbox, if I haven't got to that, I haven't got to the it's been closed notification either).

I should mention, sometimes people are CC'd because we want them to know that a task has been created, but this doesn't necessarily mean they want to know when every action happens on that task forever into the future. Having some way of distinguishing between "I need to give someone a heads up about this tasks existence" and "I want to know whenever anything happens on this task" would help here as well.

My 'useful' bar for email is "would I want to reply to this".
Does the user have to do something about this email?

I think this isn't actually anyone's bar, and isn't possible to implement anyway.

For example, here are notifications that I think many users would like to receive which they do not reply to or necessarily directly take action in response to:

• Someone landed some risky code which you reviewed.
• Someone posted a status update about a task you want to follow closely (or completed it), but it doesn't need any immediate action from you.
• Someone rejected a revision which you'd planned to review shortly (or tests failed on that revision, or the author plans changes and says they forgot some stuff).
• Someone removes you as a reviewer.

I think there are plenty of events like this where knowing about the event gives you important information to make future decisions, but does not prompt immediate action, or prompts immediate action only a fraction of the time.

T9161 is a recent example of a user wanting mail which doesn't meet this bar.

No-one wants to receive emails when someone else CC's them on a task or code review that you might only be tangentially related to.

I certainly do! And T9031 is a request explicitly asking for this behavior to override preferences! And the user CC'ing them probably wants them to receive mail, too.

Basically, for anyone who thinks the best way forward here is broadly improving the defaults, can you just enumerate what you think those defaults should be? I suspect they will be so complex and varied and contradictory from person to person that it will be clear that this is actually a very hard problem and that better defaults is, at best, a tiny part of the solution.

I also want to note the consideration that "good accuracy" + "simple ruleset" is probably a better default than "great accuracy" + "complex ruleset".

For example, suppose the defaults include mailing you revision closures if you made a meaningful interaction with the object but not if the closing commit triggered a material test suite.

This might actually be a more accurate rule than "mail you on revision closures", in that it could deliver more mail you care about and less mail you don't care about. But it's almost impossible to predict or understand, and the UI you'd end up in if you wanted to adjust it would be very complex.

A major goal of the current defaults is to have the simplest ruleset we can (approximately, "send all mail"). A solution which replaces "I get too much mail" with "I don't understand why I get mail" isn't necessarily a step forward, and almost any change will make it at least somewhat more difficult to understand why mail is sent.

So even if everyone agrees that almost no users will, say, want mail when a task is mentioned on another task (and not everyone does -- see T9284!), it isn't necessarily better to make these changes. If we get rid of 20% of mail but make the default ruleset 3x more complex, that could be worse on the balance.

14 days seemed like enough time to get some reasonable results. They are.... complicated.

• To my surprise I'm not in the top 10 for delivered email.
• Very roughly, people who get more email seem better at handling it.
• Among the 'too much email it is terrible!' crowd some people turned off virtually all email in phabricator while others have a client side filter. The most vocal complainers have usually succeeded in using the available tools to no get email, but at the cost of creating larger coworkers-ignoring-each-other ones.
• Most people are doing some filtering although perhaps ironically some of the best responders send almost everything to their client.
• For anyone looking at the raw numbers, we have a lot of audit related mail.

After looking at the table for a while I've come away without any terribly consistent conclusions. The hardest part of "handle email" seems to be the skill of handling email. Perhaps this is a skill like touch typing that "digital natives" or whatever the term of the day is in theory "Just Know" but in practice it is something you need to learn like anything else.

General product thoughts in rough order:

• As a corporate install if I could get people off on the right foot with T4103 I think that would help by reducing the initial volume people are flooded with (and help out a few lazy users). I think my personal settings are pretty reasonable ( I have turned off only a handful of boxes) and that cuts email volume by well over half on our install. I think these settings are specific to our use of audits and not ones that make sense as upstream defaults.
• Something like stackable actions (M1430) would be good on their own merits and happen to reduce email volume. I find it annoying when I get three emails in quick succession because someone (1) commented on a ticket (2) edited it (3) moved it on a workboard.
• T7013#130405 suggested only sending one message if you are subscribed to both ends of an action. I like this idea but I don't think it makes the product better in other ways besides reducing the volume of "annoying" mail.

So overall email remains a social problem without obvious technical solution. I still think it is important that the defaults are to send emails because that nudges the culture in the right direction: If a human asks you for something and CCs you, they will know about it and can then respond. I think it may be reasonable to disable self-actions by default. I had them on for a while because that was what I was used to with trac, but I rarely find them useful for anything other than testing. A few people have commented that their first time user experience with a flood of self action emails left a poor initial impression.

$ ./bin/mail volume --days 14
+=====================+============+===========+
| User                | Unfiltered | Delivered |
+=====================+============+===========+
| xxxxxx              | 950        | 950       |
| xxxxxx              | 941        | 941       |
| xxxxxx              | 860        | 860       |
| xxxxxx              | 873        | 839       |
| xxxxxx              | 1163       | 740       |
| xxxxxx              | 728        | 728       |
| xxxxxx              | 1545       | 718       |
| xxxxxx              | 599        | 592       |
| xxxxxx              | 566        | 566       |
| xxxxxx              | 560        | 560       |
| xxxxxx              | 989        | 506       |
| xxxxxx              | 503        | 503       |
| xxxxxx              | 803        | 498       |
| xxxxxx              | 1118       | 474       |
| xxxxxx              | 1243       | 424       |
| xxxxxx              | 929        | 398       |
| xxxxxx              | 583        | 397       |
| xxxxxx              | 355        | 355       |
| xxxxxx              | 1317       | 323       |
| xxxxxx              | 1109       | 314       |
| xxxxxx              | 1707       | 277       |
| xxxxxx              | 255        | 255       |
| christopher         | 1421       | 242       |

A few people have commented that their first time user experience with a flood of self action emails left a poor initial impression.

@cburroughs do you know how many users turned self-actions off? Looks like 8 of 23 people received "all mail" and maybe 10 of 23 received "mostly all mail".

~50% of the preference entries include disabling self actions. There is probably a bit of hand waving to get to that that to a percentage 'all users', or 'all active users'.

SELECT user_preferences.userPHID,user.userName FROM user_preferences JOIN user ON user_preferences.userPHID = user.phid WHERE preferences LIKE '%"self-mail":"1"%' ORDER BY user.userName;

(See also T5358 for previous discussion on this setting).

For whatever it's worth, here are the comparable (14-day) numbers from this install with the more modern script:

ubuntu@secure001:/core/lib/phabricator$ PHABRICATOR_INSTANCE=secure ./bin/mail volume --days 14 
+=========================+============+===========+
| User                    | Unfiltered | Delivered |
+=========================+============+===========+
| epriestley              | 5251       | 4249      |
| chad                    | 2338       | 548       |
...
| cburroughs              | 324        | 154       |
...

My personal, completely subjective experience is that Phabricator mail is trivial to deal with, almost all valuable, takes very little of my time, and I want to receive >95% of the mail that I receive, and that I'd barely notice if I had 10x this mail volume.

One thought is that it's potentially more practical to do special onboarding with "self mail" than with other settings: we can explain what self mail is pretty easily to users, and they don't need to understand all the applications or operations to make choices about it. It's more reasonable to ask new users if they want mail about stuff they do than to ask them if they want mail about workboard column moves, for example.

I think it's in T5358, but one particular concern I have with self mail being off by default is that self mail teaches users how mail works: it confirms that their actions went through, that they'll get mail from Phabricator, shows who else received the mail, etc. If we simply defaulted it off, we would trade some level of "too much mail" (which doesn't strand users) for some level of "how does mail work?" (which could strand users, since there's no obvious place to start if you want to figure this out).

But this option is more onboardable than other settings, and we could potentially onboard this option very gently, tell users what to expect and how to change the setting, point them at /mail/ to retroactively answer questions about what happened with mail, etc. As /mail/ improves I'm broadly more comfortable using it as a resource to try to answer user questions about mail. And ideally we'd just surface /mail/ more and not need to do special onboarding.

I think T4103 and improving /mail/ are desirable no matter what, so maybe we just have to build that stuff first and then experiment here with trying to make mail behavior obvious no matter what the install default settings are.

In T9161#136555, @epriestley wrote:

My personal, completely subjective experience is that Phabricator mail is trivial to deal with, almost all valuable, takes very little of my time, and I want to receive >95% of the mail that I receive, and that I'd barely notice if I had 10x this mail volume.

Sure, but the statistics do paint you as an outlier, both for sheer volume, and lack of (Phabricator-side) filtering. Maybe it's more helpful to look at your views and experiences in that light, rather than suggesting that others who see it as too much email have 'personal/emotional issues', which is perhaps the most unhelpful characterisation I've seen all year, and trivially flipped to the opposite position.

I think it's in T5358, but one particular concern I have with self mail being off by default is that self mail teaches users how mail works: it confirms that their actions went through, that they'll get mail from Phabricator, shows who else received the mail, etc.

Is that a useful lesson to teach? 'Here's this tool, it will send you a lot more mail than you personally are used to, requiring a fairly sizeable change in how you deal with your workflow and possibly requiring you to learn how to set up mail filters: anyway, it's all for the good, because every time you do something, you need to wait to get an email to make sure it really happend. Otherwise it might not have.'

Given that I just received a useless eadler moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board. email notification for this ticket, I suggest that T10745 be considered as part of email reduction.