Page MenuHomePhabricator

Phacility mail is going to "Spam" for some Gmail users
Closed, ResolvedPublic

Description

We've seen two reports of mail going to Gmail spam now, and this started happening to me, too.

I'm not sure how we can resolve this. Here's what we know:

  • This wasn't happening until somewhat recently (1-2 weeks ago).
  • DKIM and SPF headers haven't been changed in that time, and seem to be good (GMail adds headers implying it is happy with the SPF and DKIM settings).
  • For me, Gmail displayed a banner with wording to the effect of "this is similar to other spam messages" to explain why it was marked, but I didn't get a screenshot of the exact text.

The only thing I can come up with offhand is that we are not setting "Precedence: bulk" on these messages, but should be.

Event Timeline

epriestley claimed this task.
epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a project: Phacility.
epriestley added subscribers: epriestley, btrahan, Mnkras, warman.

Specifically, the Gmail document about this is here:

https://support.google.com/mail/answer/81126?hl=en

Some possible things we could pursue:

  • Add Precedence: bulk by default (D12266).
  • Add a List-Unsubscribe header.
  • Add a "pure" unsubscribe link which turns off all email for your account without requiring authentication (although our volume is so low that I can't imagine this is causing issues, and this creates security risks).
  • Handle bounces, although this might be hugely involved.
  • Warn users about stuff ending up in spam.
  • Possibly, users un-marking these things as spam will fix the problem on its own.
  • Try mucking around with "Sender" and "From", see T1837.
  • Pay someone who knows what they're doing (e.g., http://www.mailgun.com/managed) to navigate this mess.

Also, I suppose:

  • Finish HTML mail (T6600, etc), under the theory that just shuffling the content of mail around may help.

Two quick notes:

  • I reached out to Mailgun since that seems like the obvious place to start.
  • Our outbound email volume is growing ~30% day-over-day, so we'll be sending roughly 7e16 messages per second by the end of the year.

Is it possible that all Phabricator emails everywhere are part of this classification?

If so, I'd guess we're basically hosed as (some) users will mark these emails as spam in organizations.

I haven't seen any mail from this instance get marked as spam, and haven't seen users reporting it, although who knows.

But, yeah, that's definitely possible.

I'm getting these emails fine, I would look into the fact that all the
email stuff is for phacility.com and not admin.phacility.com

Mailgun got back to me suggesting that I check out their "Managed" offering, which is how I got in contact with them in the first place, so their sales funnel is currently looping back on itself.

While that eventually resolves, does anyone else have experience with other companies that have domain expertise who we could pay to help us figure this out rather than flailing around in the dark?

Lemme go ask some friends... :D

Mailgun's minimum Managed pricepoint doesn't make sense for us so I guess I get to learn about mail ops. :/

Steps I've taken so far:

  • I pushed the "Precedence: bulk" change to the cluster on Saturday, and here before that. I haven't seen any new deliverability issues as a result.
  • Mailgun's "Managed" program is financially out of our reach for now (we need about 100x more mail volume for it to start to make sense).
  • Mailgun had us sending through a shared IP which was blacklisted on one blacklist (SORBS-SPAM).
    • I requested a dedicated IP, but they recommend against this at our current volume.
    • They did switch us to a non-blacklisted IP.
    • This raised us from a 9/10 to a 10/10 according to https://mail-tester.com/

Personal steps I've taken:

  • I added "From" addresses to my contact list to verify that permitted delivery.
  • Then I removed them.
  • It's possible this has secret side-effects?

Effects I've seen so far:

  • Mail isn't delivered to my spam folder anymore.

Conclusions I draw from this:

  • ???

I just edited a file on admin.phacility.com and I got the email and it didn't go to spam, so thats good, @epriestley think you can have it re-send an invoice email to me? Just to make sure its working.

Yeah, give me about 30 days and I'll send you another one.

I haven't seen more reports of this in the last few weeks and haven't experienced it myself, so I'm presuming it's resolved. Here's a nice chart of outbound mail volume with the Y-axis cleverly omitted so that it looks far more impressive than it actually is:

Screen_Shot_2015-04-25_at_7.52.47_PM.png (235×660 px, 10 KB)