This issue is still occurring.. the root cause is related to phabricator being set to allow-public. My fix still works and it would be nice if anyone here would support me bringing this upstream.

One call to for-each-ref comes from resolving a list of commit hashes, to test if they still exist. This is accomplished with DiffusionLowLevelResolveRefsQuery, which calls git for-each-ref first and falls back to git cat-file --batch-check.

Has this repository changed?

A general concern with "batch processing" is that it's quite bad if one commit failing to import can stall the entire repository forever.

See also T13552, which modifies the above discussion. The "Update" steps now happen after the "Publish" step.

When you are viewing a document with engine X, and comments originally made with engine Y are present, this should be indicated ("This comment was made while looking at this change as a Q document."). They should probably also be moved to the top/bottom of the file, at least by default, since "Jupyter line number 9 = raw source line number 9" is an exceptionally bad and confusing guess at how to map line numbers.

For now, I fixed the explicit misinformation in audit.can-author-close-audit, at least.

Although I didn't look particularly hard, I can't immediately find any more evidence that this is occurring in production.

One remaining artifact here is this configuration option:

Long ago, we used to write a <package, commit> audit relationship with "AUDIT_NOT_REQUIRED", meaning "This commit affects the package, but package owners don't need to audit it.".

PhabricatorAuditStatusConstants appears to have a set of unused constants:

Evidence increasingly suggests that the root problem here was GET_LOCK() issue in T13627, not an error in an sshd subprocess context.

In general, Phacility production hosts may interact with logs as several different users:

This ran into some filesystem permissions issues and needs followup.

When we are importing a commit, we often don't have any legitimate user to act as: the commit may come from an observed repository (so we never authenticated a pusher), and the "Author" and "Committer" strings are arbitrary and untrusted in Git and Mercurial.

All three of these are almost certainly diffusion.rawdiffquery, I think the first one just predated D21463 reaching production so it didn't show the method.

This deployed, and appears resolved.

• This was originally implemented in D19357.
• This was broken by a refactoring change in D20775.
• I made problem observable in D21575 fixed it in D21576.

This is a brute force approach to support "context blocks" without taking any steps backwards (i.e., it supports older transactions and newer, modular transactions), but it feels like this is a lot of steps sideways and this change would be far better if it took a nuanced approach through T12921.

• T12921 is adjacent to mail rendering.
• T13439 mentioned the hovercard summary text.

D21574 implements the hovercard and summary length changes.

See PHI1997, which discusses things from the perspective of reporting coverage from a build system.

This more-or-less worked as desired. Some minor issues:

Update the daemon UI to break tasks down by object/container.

The affected user in PHI1980 reported that this appeared to be effective in resolving the issue.

This at least resolved the obvious badness in the case of PHI1979.

However, the existence of the original code might point at a bug in the "Variable Reused as Iterator" lint check: I would expect it to have prevented the original code in the first place.

As a coarse first pass at this, forcing the commit cache to fail results in a full discovery of the Linux repository in 14 seconds, versus 2m36s with normal cache behavior.

I updated the description, but the relevant workflow is during the "discovery" step, not the "refs" step. The "refs" step uses the RefCursor table and doesn't interact with the commit cache.

It looks like the case in PHI1977 was actually a situation of attempting to trigger an audit by writing a Differential rule, so the Global/Personal stuff may still be worth fixing but has zero known cases of actual confusion in the wild. I'm less sure how the UI could be clarified around the Audit/Differential issue.

After D21518:

These parts seem likely resolved once I convince myself the patches so far actually work:

I have a change to add containerPHID locally, but it ends up having relatively high complexity because several other patches (including 20190909.herald.01.rebuild.php) call PhabricatorRebuildIndexesWorker::rebuildObjectsWithQuery(...), which does not work if executed in sequence prior to a worker queue schema change.

This also relates slightly to T13580, but I believe the two issues are addressable independently.

However, I'd like to have a better understanding of how we're reaching this state, and I'm not satisfied that these repositories are going down the "natural" pathway (of changing ref definitions after the import starts) and suspect there is some more complicated interaction at play here.

I'm hoping to land at least a narrow fix for this today to support an import in PHI1979 tomorrow. However, I'd like to have a better understanding of how we're reaching this state, and I'm not satisfied that these repositories are going down the "natural" pathway (of changing ref definitions after the import starts) and suspect there is some more complicated interaction at play here.