As far as I know, there's no nice/easy/tradeoff-free way to fix this by, e.g., tweaking headers. We already send the correct headers and thread correctly in everything except Mail.app on Lion or newer. Starting with Lion, Mail.app more or less ignores all threading headers and primarily/exclusively uses the subject to thread mail when mail comes from different senders. Quote from Apple on this:

These messages are not threading together in Lion as they used to in Snow Leopard because we completely rewrote the threading algorithm in Lion. In Snow Leopard we always followed the headers no matter what. Although that system worked well for grouping messages like the ones described in this email thread, it failed when users replied to existing messages to start a new conversation. As a result, we added a bunch of heuristics to gauge user intention to determine how messages should be threaded. ... In order for the Facebook messages to be threaded together by the references added by the mailer, all subsequent replies need to have an Re: prefix, just as if they were composed by the senders themselves using a Mail client. Only the first message in the thread should not have an Re. If the prefix is missing, we will thread them together using the references if the senders are the same.

Basically, they broke threading of automated messages in order to improve threading of people manually using "Reply", deleting the subject, and typing a new one in its place to start a new thread. I think this is ridiculous, but didn't get very far trying to move the mountain when we got in touch with Apple via Facebook.

You can set metamta.re-prefix to true to make Phabricator put "Re:" in front of everything, which will make things work well on Mail.app (although it may break Outlook). This comes at the cost of being somewhat bad on other clients (and we must put "Re:" even on some of the initial messages the user receives, because other recipients may have part of the thread already) but might be the greater good.

You can also try disabling metamta.vary-subjects, which will make mail substantially less useful (it will no longer show the user action in the subject) but may also improve things.

If you restore metamta.one-mail-per-recipient users can alter these settings individually in /settings/, but that creates duplicate mail issues when there are direct addressees as well as mailing list addressees.

There's some additional discussion in the configuration for these options. See also discussion in T782, T847.

metamta.vary-subjects is already disabled (we don't need that).

I actually disabled metamta.re-prefix because of the problem that the ininitial review mails had Re: in them, which I found to be a really bad experience.

In our case, the messages are coming through a public list, so I can't enforce my preferences on everybody.

Why does metamta.re-prefix use the prefix for initial mails? Surely it can tell that this is the first time a patch is being uploaded for review?

I think we could drop it on some initial emails -- there are some cases where you're added as a reviewer or whatever and we need to add "Re:" even though we know it's the first mail we're sending to you, because you may already be part of the thread through membership in a mailing list addressee. Basically, we have the concept of "first mail to recipient" but not "first mail in thread", which is why we add "Re:" unconditionally. But I think we can improve the behavior in at least some cases.

To keep everybody in the loop.
After a irc conversation, I applied the following patch:
https://github.com/r4nt/phabricator/commit/a787defb64abd16bc3321a38ff827e3e6d15b65f

which solves this problem for us - this will still send Re: when it's the first mail to a user who is cc'ed, but in our case all users that are involved in code reviews are cc'ed via a common mailing list anyway, so for us this is actually always the right thing to do (if the mail was already cc'ed to the user, but doesn't have the Re:, apple mail would again thread incorrectly, as far as I can tell).

With that patch, I get incorrect threading in Mail.app under various configurations:

{F21755}

Are your Mail.app users reporting it working?

Yep, they're reporting it's working...

Any chance to get this in? Perhaps under and option? Eitan, how are you guys in FreeBSD land handling this?

Please see Planning and Contributing Code. See also T8227.

Awesome docs, thanks!

The big question before I can either send you a patch or try to get funding to pay you for the prioritization is whether you think you'll accept the feature. So let me get more concrete if that helps:

• an option is apparently probably not going to fly according to your reasoning (which I generally agree with :)
• if we're not doing an option, Re: behavior would change significantly; this seems risky
• a different design might be to make mails more configurable via extensions (perhaps they are already and I am unaware of how to do this?)

No one's mentioned anything to us about this. I suspect most of our users are not using Apple Mail though.