@fooishbar is the only user I could find here that might know what's happening over there. maybe?

In the case of the user we're familiar with from elsewhere, it looks like this is what happened:

He, or someone else with his email address at the time, reported an issue against Gnome in 2009: https://bugzilla.gnome.org/show_bug.cgi?id=570762

Recently, someone associated with Gnome/Freedesktop imported Bugzilla into their Phabricator instance. For example, here's the imported version of the above bug:

https://phabricator.freedesktop.org/T2021

It looks like only bugs related to "Pitivi" were imported.

The import process created accounts for users and sent them welcome mail. This happened to coincide with T9046, so none of the links in the welcome mail worked.

I don't think these users are getting "ALL" the emails -- just a second copy about any event in the last 5 years. For active users this could be a lot of mail, but it's hard to tell without knowing how the import process worked, exactly.

So ... do we send emails if an account hasn't been verified? Or did they import around that?

We have to send the initial welcome mail for unverified accounts, since obviously you can't invite new accounts without sending them an initial email.

We send additional mail for unverified accounts in general because it's desirable in the common case: you're a new employee, so I create an account for you and add you to some tasks, and you get email about it immediately. When you log in a little later, you don't have to catch up on anything you missed. We could change this behavior, but it would be worse in the common case and better only in the case where an administrator made a error (which is usually extremely rare).

Usually, the recipient can follow the welcome link and disable mail for the account if they don't want to receive it. Since this coincided with T9046, the welcome links didn't work. Once T9046 is fixed on the server, the affected users can do password resets to gain access to their accounts, then adjust settings, but obviously that's pretty cumbersome.

Another thing we don't do (but could) is have a no-auth unsubscribe link in mail. One concern with this is that it lets an attacker disable your email with a much lower barrier than authenticating (e.g., they do not need a MFA token) and then potentially escape detection longer than they otherwise would. A second concern is that the number of users unsubscribing by accident and getting confused by it might be much larger than the number of mistaken accounts. We could defuse the "attacker" case by having a "do not allow my account to unsubscribe" option, or maybe by only activating "unsubscribe" for unverified accounts. This might help with the second case too. However, it's a moderate amount of work and complexity to do this and it hasn't really come up before.

Even if we had all this, we'd still get some level of user confusion: we got at least one direct report after a user received a single email they didn't like. No matter how conservative we are, we have to send one email for "invite / create" to work.

I think the ultimate error here lies with an underdeveloped import process, and that the only real attack we can pursue is trying to make it easier for administrators to write better import processes (T3179). I can't imagine a human actually tested this and thought it was good to go, since getting copies of all transactions for the last 10 years with zero context is obviously going to confuse/frustrate a lot of users.

Funny, I was going to mention having real importers might resolve this.

I guess I don't really see any good ways forward here from the upstream perspective.

We could add some kind of "disable this address" link to mail sent to unverified addresses, but it's quite a bit of work/complexity, won't undo the damage that's already done, and may not prevent any future damage. In the best case, this only slightly mitigates issues. Without T9046, users could have clicked the welcome link and then gone to Email Preferences anyway, which is only a bit more complex than a "disable this address" feature would have been. I think this is still maybe-reasonable to build at some point, but really only makes the situation very very slightly better.

We could make the default rules more conservative, and only send welcome/verify mail to unverified addresses (for non-primary addresses, we already do this). This is fairly straightforward, but users would still receive at least one mail (which is enough to upset some of them) and this is a bit worse for the other 99% of new users. I'm not sure if it's a good tradeoff to make things worse for most users in order to slightly reduce the damage caused by administrative mistakes.

We could build pieces of T3179, but this is a huge amount of work/complexity and I don't like prioritizing things just because it's possible to make mistakes with them.

We could write a "pitfalls in importing" document or something, but it would all boil down to common sense I think ("test your importer a lot before you run it!!") and I think basic common sense steps were either accidentally missed or just not considered carefully here, so I'd guess that more warnings in the documentation wouldn't have helped.

My only random thought is if email auth is open and the install is public, not sending email until you've been approved. This is mainly an Open Source host issue. I hesitate at suggesting an admin level meta-mta option also.

T2943 is likely going to come up as well.

My only random thought is if email auth is open and the install is public, not sending email until you've been approved. This is mainly an Open Source host issue.

I think this is probably the best short-term solution to resolve the issue occurring in future.

(non open source installs are far more likely to have an administrator reachable within an organisation that can either be alerted or let other employees know what's happening)

@chad Thanks; that is indeed our instance. The import from Bugzilla in this case is done by @thiblahute, who can shed some more light on it, but I have some ideas ...

Some people specifically mentioned getting belted by project-unsubscription mails in a horrific feedback loop: someone leaves a project, which mails everyone, which prompts more people to leave, etc. I'm looking into how to forcibly change everyone's settings to elide project-join/leave mails [edit: done] (and manually letting the few people who do care about those know), as well as changing the defaults [edit: this seems difficult/tedious, but, D5636]. Any quick pointers welcome, but I guess it shouldn't take long.

The above is kind of a clue: the import had added everyone who'd ever reported or commented on a bug, to the project, which was then CCed to basically all activity. Obviously this wasn't the right thing to do.

Coinciding with T9046 was obviously pretty brutal [edit: fixed].

For what it's worth, this has been using a very rough set of scripts to populate Phabricator from a number of sources; Phill imports from a JSON + external attachments set, which is generated by external scripts that we currently have for Bugzilla and JIRA.

https://git.collabora.com/cgit/user/em/phabricator.git/tree/scripts/phill?h=phill
https://github.com/thiblahute/bztophill

Many apologies for the PR disaster.