The stalled transactions on this host published after I deployed the update.

This happens when a recipient list includes an Owners package which has been destroyed. Specifically, we'll exit this section of PhabricatorMetaMTAMemberQuery with out the PHID in $package_map, and then fail to return it:

I reordered these elements in D21200. See also D20967.

I think that might be everything? Not entirely sure, but haven't seen any more since the last deploy.

I deployed that last round of things to secure. Not totally confident I got everything, but hopefully we're in better shape now.

uhoh looks like I'm a dum dum

This is doing somewhat better now, but I've still seen:

D20563 probably fixes mail. I don't think (?) it will fix notifications but haven't hunted that down yet.

Per D20533, the major query this UI uses is currently unkeyed (no dateCreated key on transaction tables).

My expectation is that this is effectively resolved by T13277. Since "Autoclose" is no longer a separate permission from "Publish", commits should now always mention + close or never mention + close.

quack

• D20208 adds an authorPHIDs constraint.
• D20209 provides details for transactions which add or remove projects. See above for format and rationale, or just query something and see what pops out.
• D20211 makes the documentation less bare-bones.

Currently, project tag changes and subscriber changes don't expose data in "transaction.search" because there's no "obviously correct" way to represent these changes in a future-proof way.

To my surprise, it also seems the method does not output any project tag changes that happened on a task?

Thanks for the quick reply!
Haven't played with webhooks yet so I did not know/realize it's transaction PHIDs only - "PHID" sounded generic enough that I incorrectly expected any kinds of PHIDs to be accepted as input. So this task is somewhere between either "clarify documentation that only transaction PHIDs are supported" or a "support other PHIDs" low-prio enhancement request.

The phids constraints matches transactions with specific PHIDs (PHID-XACT-...). Usually, you'll have a list of transaction PHIDs when you're responding to a webhook callback, so the phids constraint is most useful for webhooks.

Probably a good idea, but not worth keeping a task around for.

I believe we haven't seen more of this in two years, and "make the worker always exit in less than 2 hours" is a more-or-less reasonable remedy. Getting one extra email every two hours also isn't a huge problem even if we do get this wrong.

I believe D19969 has now fixed this, although I'm not entirely certain, since it was never reliably reproducible in production so there's no way to really test or verify it.

This isn't trivial to resolve. The inverse transaction goes through standard "old value / new value" logic, so if we just move the entire "apply inverse transactions" block to later on, the transactions automatically no-op themselves: they do nothing by the time we apply them.