• I think this sends the recipients many copies of the email, particularly if they are on mailing lists? Why do you believe that users will receive only one copy of the mail if we send ten messages with their address in "To"?
• Are you sure "Reply-To" overrides "Reply All" in all major clients? I haven't tested this personally, but I can't immediately find any references to this behavior on Google. I believe we already use "Reply-To", so I would expect "Reply All" to already be a non-problem if "Reply-To" overrode it. My pre-existing belief was that "Reply-To" replaced only "From", not the entire recipient list, under "Reply All".

Or, more generally, if we send two mails:

To: X, Y
Reply-To: Secret-Address-For-X

To: X, Y
Reply-To: Secret-Address-For-Y

...what guarantees that user X receives only one of them, and that the one they receive is the one with the right secret?

Unless I'm missing something, I think the instructions here cause X to receive two mails (they are on "To" for both).

If some header like Message-ID could de-duplicate them on the client (or in the mailbox adjacent to the client), they might only see one in their mailbox, but nothing would guarantee that X and Y saw the versions with the right secrets -- they'd probably see whichever one arrived first? (Presumably, usually the one we sent first.)

Also note that we can not control Message-ID (or some other headers) with some mail providers (notably, SES).

I don't think there's a header like Ignore-The-"To"-Header-And-Only-Send-To-These-Recipients: X that lets us lie in To but really send to a different recipient list, but that seems like it's required for this approach to be viable.

Unless I'm missing something, I think the instructions here cause X to receive two mails (they are on "To" for both).

This is the difference between the envelope TO and the header TO. Here is an example

Here is how I sent it

∴sendmail -i -- mymail@my-real-domain.example.com < to-eax
[44007 12:51:03.343 eax@tw-mpb-eax ~/exps/mail]∴cat to-eax
From: yelo@website.com
To: someone@important.com
Subject: Yellow Blosums 2

EOF

Some more thoughts: even if we can't control the Reply-to in all cases, phabricator can look at the TO header and perhaps choose not to send mail if it looks like it was already sent to a given user. Of course, given the above, this is spoofable.

From: yelo@website.com
To: someone@important.com, bob@bob.com
Reply-to: only-me@somewhereelse.com
Subject: Yellow Blosums 2

EOF

works for me on some mail clients: both Reply and Reply All send only to only-me@somewhereelse.com. On other clients Reply works but Reply-All does not (it sends it to everyone, including the reply-all address). We may not be able to eliminate all the options then, but I do think we can get closer to being better and at least not violating policy when using this option.

Oh, I didn't even realize that was a thing. I assumed SMTP went like "okay here is a message: <message body>".

I don't think we have control over the SMTP envelope for at least some of of the APIs we implement, although perhaps I'm mistaken. In some cases I chose the non-SMTP version of things (e.g., POST some JSON) when an SMTP version was available since I mostly know how JSON works but do not mostly know how SMTP works. It's possible we could revisit those decisions and do more SMTP to get greater access to envelopes, although I'd guess not all adapters provide us envelope access.

Some more thoughts: even if we can't control the Reply-to in all cases, phabricator can look at the TO header and perhaps choose not to send mail if it looks like it was already sent to a given user.

We already attempt to do this -- are you seeing it not work in practice? Rough pathway is:

• PhabricatorMetaMTAReceivedMail loads all objects associated with addresses which were present in the "To" or "CC" fields of the mail.
• These are passed to the response via setExcludeMailRecipientPHIDs().
• This mail is dropped with PhabricatorMetaMTAActor::REASON_RESPONSE:

'This message is a response to another email message, and this '.
'recipient received the original email message, so we are not '.
'sending them this substantially similar message (for example, '.
'the sender used "Reply All" instead of "Reply" in response to '.
'mail from Phabricator).'

To step back here, what are the concrete issues you're running into with one-mail-per-recipient?

In T12240#211108, @epriestley wrote:

Oh, I didn't even realize that was a thing. I assumed SMTP went like "okay here is a message: <message body>".

I don't think we have control over the SMTP envelope for at least some of of the APIs we implement, although perhaps I'm mistaken. In some cases I chose the non-SMTP version of things (e.g., POST some JSON) when an SMTP version was available since I mostly know how JSON works but do not mostly know how SMTP works. It's possible we could revisit those decisions and do more SMTP to get greater access to envelopes, although I'd guess not all adapters provide us envelope access.

Some more thoughts: even if we can't control the Reply-to in all cases, phabricator can look at the TO header and perhaps choose not to send mail if it looks like it was already sent to a given user.

We already attempt to do this -- are you seeing it not work in practice? Rough pathway is:

'This message is a response to another email message, and this '.
'recipient received the original email message, so we are not '.
'sending them this substantially similar message (for example, '.
'the sender used "Reply All" instead of "Reply" in response to '.
'mail from Phabricator).'

We don't have public-reply-to turned on because we are worried about reply-to spam. In particular the belief is that turning on public-reply-to will cause duplicate emails. If this is not the case we can likely just change this setting.
The text that is causing us to believe this is:

Sometimes people will "Reply All" and everyone will get two mails, one from the user and one from Phabricator turning their mail into a comment.

To step back here, what are the concrete issues you're running into with one-mail-per-recipient?

some of the problems are concrete and known and thus solvable.

This violates policy controls. The body of the mail is generated without respect for object policies.

As a result of having public reply-to turned off, and metamta.one-mail-per-recipient turned off is that there is no way to reply to the email and phabricator process it at all.

To jump back further, the root-ier cause is that you're using mailing lists?

If we implemented T3980, could you create projects for mailing lists instead of "Mailing List" users, then do one-mail-per-recipient + private-replies and be generally OK?

Sometimes people will "Reply All" and everyone will get two mails, one from the user and one from Phabricator turning their mail into a comment.

I think this is mostly just out of date (I wrote it before we wrote the "exclude recipients" stuff) but you'll still probably at least get some cases like this:

User [Reply-All]: !acpetpt looks great
Phabricator [To User]: Sorry, that's not a valid command.
User [Reply-All]: !accept looks great

(D17331 should fix the outdated discussion of the option in Config.)

In T12240#211112, @epriestley wrote:

To jump back further, the root-ier cause is that you're using mailing lists?

If we implemented T3980, could you create projects for mailing lists instead of "Mailing List" users, then do one-mail-per-recipient + private-replies and be generally OK?

We actually implemented this ourselves and moved back to mailing list users. One issue with one-mail-per-recipient is that gmail does not permit muting an email conversation if you are the only TO recipient. Another issue is that it makes filtering mail to specific mailing lists impossible if you don't have the ability to filter on arbitrary headers. Sadly a lot of this problem is due to the inferiority of the gmail email client.

FWIW I think implementing part of this proposal would at least solve this issue:

Policy controls work correctly and are enforced per-user.

so even if my root cause can't be fixed, this should remain open.

One issue with one-mail-per-recipient is that gmail does not permit muting an email conversation if you are the only TO recipient.

I don't necessarily want to do actually this, but could we hypothetically fix this in a technical sense by adding a dummy recipient to every message ("Dummy-Recipient-So-Gmail-Will-Let-You-Mute-This@gmail-mandated-by-intergalactic-law.com")?

Sadly a lot of this problem is due to the inferiority of the gmail email client.

Perhaps now that we have a new administration, Trump will overturn the galactic law mandating all users use Gmail and the Gmail web client for all email.

If we use SMTP envelope To, doesn't Alice potentially get multiple messages still (envelope "To: Alice", envelope "To: All-About-Alice@lists.company.com")? The list rewrites the envelope to go to all the list members, right? Or is there more magic there I don't know about?

We could also implement "Mute Thread" in Phabricator, but I assume no one would be willing to click twice to mute a thread.

(dropped this since I don't have time to fully test, but I might come back to this in the future)

In T12240#211216, @epriestley wrote:

One issue with one-mail-per-recipient is that gmail does not permit muting an email conversation if you are the only TO recipient.

I don't necessarily want to do actually this, but could we hypothetically fix this in a technical sense by adding a dummy recipient to every message ("Dummy-Recipient-So-Gmail-Will-Let-You-Mute-This@gmail-mandated-by-intergalactic-law.com")?

I think so. I've considered implementing a local patch to try this too. :)

In T12240#211217, @epriestley wrote:

If we use SMTP envelope To, doesn't Alice potentially get multiple messages still (envelope "To: Alice", envelope "To: All-About-Alice@lists.company.com")? The list rewrites the envelope to go to all the list members, right? Or is there more magic there I don't know about?

With appropriate message-ids this shouldn't be a concern but if don't have control over that then it still becomes an issue. I'd need to spend some time with different clients and different messages to fully test :(

I'm pretty sure we can't generate our own Message-ID with Amazon SES, at least -- although we use the HTTP API, not the SMTP API, and maybe the rules differ. Not sure about other providers.

In T12240#211220, @epriestley wrote:

We could also implement "Mute Thread" in Phabricator, but I assume no one would be willing to click twice to mute a thread.

In a monorepo the size of ours, you'd be surprised. Large changes still have to go through review for compliance reasons, even though you guys say things like renames should not be reviewed in the context of a Differential.

This also comes into play with a feature request that we have for "Resign from this review, even though I'm in an owners package, and stop spamming me" but there's more to come on that.

We'll probably implement a Phabricator-side mute-this-thread in connection with T10448, I'm just guessing that some of the users complaining that GMail mute doesn't work won't consider clicking a link in the email body, then clicking "Mute Thread" to be an acceptable solution.