When sending mail via SMTP, the outbound pathway may be something like SMTP → SES. In this case, we can't set a "Message-ID" header: even though it may be respected by the local MTA, it will be discarded by SES.
Today, we'll incorrectly believe that we can set a "Message-ID" header: we can with vanilla SMTP servers, and configuring a problematic service via SMTP is somewhat unusual because we have direct bindings for the most popular ones. Still, it's reasonable to configure them via SMTP instead.
When we guess wrong about whether we can set "Message-ID", we get less-than-optimal threading behavior.
To allow these mailers to be configured so we guess right, I'm planning to:
- Add a true/false/null flag to the SMTP and sendmail mailers, called something like 'supports-message-id'.
- A value of true or false explicitly configures our detection of supportsMessageIDHeader().
- A value of null (default) causes us to guess based on the SMTP hostname. This doesn't have to be terribly exhaustive/reliable, but we can get it right in some reasonable cases (e.g., if we're doing SMTP directly to SES), and problems here are generally difficult to detect/identify (they look like "threading is bad", which is many steps removed from "configure an obscure secondary option in your outbound mailer setup") so there's probably enough value in this to justify a list of regexps living in the codebase.
We have upgraded to 2019 Week 8. Gmail, even when "Vary Subjects" is turned off, now threads the first email about a Differential revision in a separate thread from all subsequent emails about the same revision. I believe the difference is that the first email no longer specifies a (fictional) In-Reply-To identifier.
Before the upgrade:
# First email Message-ID: <email@example.com> In-Reply-To: <differential-rev-PHID-DREVfirstname.lastname@example.org> # Second email Message-ID: <email@example.com> In-Reply-To: <differential-rev-PHID-DREVfirstname.lastname@example.org>
This previous behavior violated standards — the In-Reply-To identifier is supposed to be the Message-ID of a previously delivered email — but at least it wove a consistent fiction for mail clients. But following the upgrade:
# First email Message-ID: <email@example.com> # Second email Message-ID: <firstname.lastname@example.org> In-Reply-To: <differential-rev-PHID-DREVemail@example.com>
I vaguely suspect b5797ce60a35e1b5d395598dec5dc22d57288120 but am not at leisure to actually do any bisection today (or to figure out the new email order of operations to learn when the header gets added).
Would it be possible either:
- For the fictional In-Reply-To to be restored to the initial email in each series.
- Or, for the initial email to have as its Message-ID the same identifier that all subsequent In-Reply-To's will specify?
Either tweak would get threading working again for our myriad developers.
While I'm typing, I should share that the fusillade of tags that Phabricator now attaches to the bottom of every email have revolutionized our ability to use Gmail's weak filtering system with Phabricator notifications, and has made many developers much happier with their experience! Thanks for implementing that.