Page MenuHomePhabricator

Improve implementation of "Re:" prefix by dropping it from messages which are unambiguously first-in-thread
Closed, InvalidPublic


I got a report from one of our users of the open source mailing list that phab's mail threading breaks for Apple Mail's "Organize by Conversation". There are enough people on the list that use Apple Mail that this is really important for us to get right. I'm happy to help investigating.

"""the e-mails that go to cfe-commits don't seem to get coalesced by Apple Mail's "Organize by Conversation" feature, even though they have the same subject."""

"""Quickly scanning over my inbox shows none of them working -- or rather, it's grouping them if they have the same sender, but not otherwise. So for "[PATCH] Add script to update AST matcher reference", I see your e-mails in one group and Dmitri's in another. I'm going to backpedal on what I said previously because I /do/ see "In-reply-to" and "References" fields in both e-mails now (not sure how I missed that, or maybe it's new). So now I'm confused.


I hope it's not trying to do something intelligent with the Thread-index header and failing, or simply choking on the long reply message name. I'll file a bug on our side too and see who figures it out first.

Event Timeline

As far as I know, there's no nice/easy/tradeoff-free way to fix this by, e.g., tweaking headers. We already send the correct headers and thread correctly in everything except on Lion or newer. Starting with Lion, more or less ignores all threading headers and primarily/exclusively uses the subject to thread mail when mail comes from different senders. Quote from Apple on this:

These messages are not threading together in Lion as they used to in Snow Leopard because we completely rewrote the threading algorithm in Lion. In Snow Leopard we always followed the headers no matter what. Although that system worked well for grouping messages like the ones described in this email thread, it failed when users replied to existing messages to start a new conversation. As a result, we added a bunch of heuristics to gauge user intention to determine how messages should be threaded. ... In order for the Facebook messages to be threaded together by the references added by the mailer, all subsequent replies need to have an Re: prefix, just as if they were composed by the senders themselves using a Mail client. Only the first message in the thread should not have an Re. If the prefix is missing, we will thread them together using the references if the senders are the same.

Basically, they broke threading of automated messages in order to improve threading of people manually using "Reply", deleting the subject, and typing a new one in its place to start a new thread. I think this is ridiculous, but didn't get very far trying to move the mountain when we got in touch with Apple via Facebook.

You can set to true to make Phabricator put "Re:" in front of everything, which will make things work well on (although it may break Outlook). This comes at the cost of being somewhat bad on other clients (and we must put "Re:" even on some of the initial messages the user receives, because other recipients may have part of the thread already) but might be the greater good.

You can also try disabling metamta.vary-subjects, which will make mail substantially less useful (it will no longer show the user action in the subject) but may also improve things.

If you restore users can alter these settings individually in /settings/, but that creates duplicate mail issues when there are direct addressees as well as mailing list addressees.

There's some additional discussion in the configuration for these options. See also discussion in T782, T847.

metamta.vary-subjects is already disabled (we don't need that).

I actually disabled because of the problem that the ininitial review mails had Re: in them, which I found to be a really bad experience.

In our case, the messages are coming through a public list, so I can't enforce my preferences on everybody.

Why does use the prefix for initial mails? Surely it can tell that this is the first time a patch is being uploaded for review?

I think we could drop it on some initial emails -- there are some cases where you're added as a reviewer or whatever and we need to add "Re:" even though we know it's the first mail we're sending to you, because you may already be part of the thread through membership in a mailing list addressee. Basically, we have the concept of "first mail to recipient" but not "first mail in thread", which is why we add "Re:" unconditionally. But I think we can improve the behavior in at least some cases.

epriestley renamed this task from Phabricator e-mails not threading nicely in Apple Mail to Improve implementation of "Re:" prefix by dropping it from messages which are unambiguously first-in-thread.Aug 31 2012, 1:15 PM
epriestley triaged this task as Normal priority.

To keep everybody in the loop.
After a irc conversation, I applied the following patch:

which solves this problem for us - this will still send Re: when it's the first mail to a user who is cc'ed, but in our case all users that are involved in code reviews are cc'ed via a common mailing list anyway, so for us this is actually always the right thing to do (if the mail was already cc'ed to the user, but doesn't have the Re:, apple mail would again thread incorrectly, as far as I can tell).

With that patch, I get incorrect threading in under various configurations:


Are your users reporting it working?

Yep, they're reporting it's working...

epriestley changed the visibility from "All Users" to "Public (No Login Required)".

Any chance to get this in? Perhaps under and option? Eitan, how are you guys in FreeBSD land handling this?

Awesome docs, thanks!

The big question before I can either send you a patch or try to get funding to pay you for the prioritization is whether you think you'll accept the feature. So let me get more concrete if that helps:

  • an option is apparently probably not going to fly according to your reasoning (which I generally agree with :)
  • if we're not doing an option, Re: behavior would change significantly; this seems risky
  • a different design might be to make mails more configurable via extensions (perhaps they are already and I am unaware of how to do this?)

No one's mentioned anything to us about this. I suspect most of our users are not using Apple Mail though.

eadler added a project: Restricted Project.Jul 25 2016, 4:55 PM
eadler moved this task from Backlog to Nits on the llvm board.

This is years old and I have no idea how to reproduce it. (I use Mail with "Organize by Conversation" as my primary mail client, and have for about a decade.)