Send email with recipient's language and access levels, not sender's
Closed, ResolvedPublic

Description

Currently when an email is sent, we send it in the originating user's locale. We should sent it in the recipients.

Repro:

  • Set User1 to ALL_CAPS
  • Assign task to User2
  • User2 gets email in ALL_CAPS

Expectation:

  • User2 gets email in their locale

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes

There's some code which makes an effort to do this already, although I think it never really got hooked up. A few general things here:

  • If metamta.one-mail-per-recipient is not set, we can't do anything about this. The option instructs us to build a single mail with a large "To/Cc" instead of sending one mail to each user. Since we're building only one mail, we can only write it in one language. We should probably always send in the server's default language, although I think there's some logic right now which tries to pick some other language if every recipient has that language as their selection. That seems complex/inconsistent/not-very-useful to me and I think we should probably just scrap it and always choose the server default. For example, I could imagine it being frustrating if I received some mail in English and some in Spanish and couldn't search through it (or write mail rules) consistently. Even if I'm primarily a Spanish-language user, getting everything in English seems better than a mixture of English and Spanish if the install I'm interacting with is primarily an English-language install.
  • A second, more severe issue than the locale issue is that we use the sender's view permissions to build every email, even when sending one mail to each recipient. In particular, this means that users CC'd on a task can see the title of revisions or tasks which are attached to the task, even if they can not see these titles in the web UI. For example, Alice (a low-permission user) is CC'd on T123. Bob attached T124: Fire Alice (a task which Alice does not have permission to view) to T123. In the web UI, Alice can see that Bob attached a task, but can not see the title of the task (which is the correct/expected set of things Alice is permitted to know). However, the email she receives generates with Bob's permissions and language, so she can see the title of the task there.
    • Since Alice already needs to be CC'd, this only leaks titles, and this is also unavoidable with metamta.one-mail-per-recipient off, which is somewhat-common, I haven't considered it too important from a security point of view. It's probably the biggest remaining inconsistency in the policy model that I'm aware of, though (now that we've fixed the Files stuff almost-completely).
  • We currently build email bodies in-process in TransactionEditor. Although we could make that logic more complicated and handle the language/security stuff there, I think we may be better off moving mail construction to the daemons. So TransactionEditors would build a list of recipients and a list of transactions, and queue some daemon task to actually handle the details of building a mail message for each user and then queuing the actual send task. This would make the logic a bit more like Feed stories, where the actual story is just a recipient list and a transaction list and then the story is built later on out-of-process.
epriestley renamed this task from Send email in recipients language, not senders to Send email with recipient's language and access levels, not sender's.Oct 22 2014, 1:47 PM
epriestley raised the priority of this task from Low to Normal.
epriestley added a project: Security.
epriestley added a subscriber: hach-que.

@hach-que and I talked about the policy half of this on IRC recently -- here's a formal task if you want to keep an eye on it.

revi added a subscriber: revi.Oct 25 2014, 2:00 PM
qgil added a subscriber: qgil.Dec 3 2014, 12:24 PM

This is the upstream ticket tracking https://phabricator.wikimedia.org/T84941.

See also T4411 for some discussion of why I know that, although I should not.

Actually, T4411 has come along far enough that I wouldn't know that at HEAD.

aklapper moved this task from Backlog to Important on the Wikimedia board.
eadler added a subscriber: eadler.May 22 2015, 4:22 PM

These Editors are the only ones with state properties:

  • DifferentialTransactionEditor
  • DifferentialDiffEditor
  • LegalpadDocumentEditor
  • PhabricatorAuditEditor
  • PholioMockEditor
  • PhortuneCartEditor
  • PhrictionTransactionEditor

It's possible that other Editors implicitly put some state on the object or squirrel it away in secret places, but I'll focus on these first.

Not actually meaningfully stateful:

  • PholioMockEditor: No state affects publishing.
  • DifferentialDiffEditor: No state affects publishing.
  • LegalpadDocumentEditor: State only used prior to publishing. We could move this state to applyFinalEffects() to make it more clear that the Editor isn't really stateful, but I'm not going to bother for now.

Stateful:

  • PhabricatorAuditEditor: Stateful on affected files and patches. This is probably justified by performance concerns.
  • PhortuneCartEditor: Stateful on invoice errors. This is under-architected but not a gaping wound.
  • PhrictionTransactionEditor: Very slightly stateful on the content diff URI.
  • DifferentialTransactionEditor: Stateful on changedPriorToCommitURI.

This change is potentially going to make mailing lists useless for nonpublic installs. Our immediate options are:

  • Use a logged-out viewer: this is correct, but mailing lists will generally stop receiving mail.
  • Use an omnipotent viewer: this will keep the mail flowing, but make this issue even worse.
  • Use the actor: this would pretty much maintain the status quo of violating policies (but only a little bit).

I suspect the best fix here may be to convert mailing lists into real users (similar to bot users). They would be unable to login or access the Conduit API, but could belong to projects, have access to Spaces, etc. This also resolves various outstanding tasks related to mailing lists.

epriestley added a comment.EditedJun 2 2015, 2:42 PM

I suppose using a logged-out viewer might be fine, since we'll use the omnipotent viewer if metamta.one-mail-per-recipient is not set, and essentially all installs that use mailing lists probably have this setting configured off.

avivey removed a subscriber: avivey.Jun 3 2015, 9:04 PM

One thing I caught is that things which use Transactions now need to implement ApplicationTransactionInterface. At least one object (PhameBlog) does not. I'll check the rest of the codebase and see if there are any more.

As far as I can tell, we're missing these:

  • NuanceItem, NuanceRequestor, NuanceQueue
  • PhabricatorUser
  • PhameBlog
  • PhortunePaymentProviderConfig

Only PhameBlog actually sends email, so I'll leave the others for when we do something with them that needs the interface.

epriestley closed this task as Resolved.Jun 4 2015, 2:31 AM

This seems to be working correctly in HEAD. In particular:

  • In both "one mail per recipient" and "one mail to all recipients" modes, recipients who do not have access to see objects will no longer receive mail about them.
  • Mailing lists are now users, and subject to whatever policy constraints those users are subject to.
  • In "one mail per recipient" mode, the mail is now built with the recipient's language settings and view policies.
  • In "one mail to all recipients" mode, the mail is built with the server's default language settings and the omnipotent (policy-volating) viewer. (This happens even if the mail only has one recipient, to keep threads in a consistent language and policy context.)
    • Because this option can violate policies, it can no longer be configured from the web UI.
    • The help for this option now warns administrators that it violates policies.

These changes were fairly sweeping, so please let me know if you see anything suspicious with mail/feed/notifications.

Is it expected that I still receive "Unknown Object" emails? I thought that this task would've solved this issue. Specifically, I received the following email from this install:

Date: Sun, 14 Jun 2015 13:55:43 +0000
To: REDACTED
From: "epriestley (Evan Priestley)" <noreply@phabricator.com>
Reply-to: T5681+27992+02827ca77acb1b46@phabricator.com
Subject: [Maniphest] [Updated] T5681: Allow applications to provide custom policy rules
Message-ID: <0000014df25ac266-554864a5-8de8-4081-9d04-61a39ebd59af-000000@email.amazonses.com>
X-Priority: 3
Thread-Topic: T5681: Allow applications to provide custom policy rules
X-Herald-Rules: <80>, none, <107>
X-Phabricator-Projects: <#freebsd>, <#policy>
X-Phabricator-To: <PHID-USER-ba8aeea1b3fe2853d6bb>
X-Phabricator-Cc: <PHID-USER-euuoczx6gv6qgx27peft>
X-Phabricator-Cc: <PHID-USER-5pbnbdnulu2hjfmqkmaw>
X-Phabricator-Cc: <PHID-USER-jmvolb7vo5qolixkccuq>
X-Phabricator-Cc: <PHID-USER-ees56kwwguslhqwe254y>
X-Phabricator-Cc: <PHID-USER-462xomdaog6qjw4uockx>
X-Phabricator-Cc: <PHID-USER-we4qnnjk4hiq33xcuov6>
X-Phabricator-Cc: <PHID-USER-vg42yoljioxdv5iac3py>
X-Phabricator-Cc: <PHID-USER-p2axfzcmoluit7llpzt2>
X-Phabricator-Cc: <PHID-USER-nbueerxdfl6csylnv6oe>
X-Phabricator-Cc: <PHID-USER-klzgd7tw6zaviykrts7j>
X-Phabricator-Cc: <PHID-USER-ba8aeea1b3fe2853d6bb>
X-Phabricator-Cc: <PHID-USER-7avs2zqt5vj35kcwi6jl>
X-Phabricator-Cc: <PHID-USER-3yc34eijivr6rqs4vgiw>
In-Reply-To: <maniphest-task-PHID-TASK-jdqcqjrzs27dbjht2asz@phabricator.com>
References: <maniphest-task-PHID-TASK-jdqcqjrzs27dbjht2asz@phabricator.com>
Thread-Index: OTYyMmNjMGRmOWJlZTk3YjVjMzU3ODgwNDMyIFV9h98=
Precedence: bulk
X-Phabricator-Sent-This-Message: Yes
X-Mail-Transport-Agent: MetaMTA
X-Auto-Response-Suppress: All
X-Phabricator-Mail-Tags: <maniphest-other>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/html; charset="utf-8"
X-SES-Outgoing: 2015.06.14-54.240.9.118
Feedback-ID: 1.us-east-1.pGOpYxVIMAH0SQCvAG4joaSeOLUocwRUW94fFoAvTwo=:AmazonSES

<div>epriestley added a commit: Unknown Object (Diffusion Commit).</div><br /><div><strong>TASK DETAIL</strong><div><a href="https://secure.phabricator.com/T5681" rel="noreferrer">https://secure.phabricator.com/T5681</a></div></div><br /><div><strong>EMAIL PREFERENCES</strong><div><a href="https://secure.phabricator.com/settings/panel/emailpreferences/" rel="noreferrer">https://secure.phabricator.com/settings/panel/emailpreferences/</a></div></div><br /><div><strong>To: </strong>epriestley<br /><strong>Cc: </strong>20after4, robclancy, eadler, devurandom, Krenair, mikaye, aklapper, qgil, chad, sergey.vfx, epriestley, sascha-egerer, joshuaspence<br /></div>

I wouldn't expect this to have changed that -- see T4345.

urzds added a subscriber: urzds.Jul 12 2017, 11:14 AM