See PHI1304. An install with a large number of observed repositories is reporting intermittent update delays on rarely-used repositories.
A fishing expedition failed to find a smoking gun, and it's possible this is something very obscure like "0.001% of the time, git symbolic-ref hangs indefinitely" that we'd have a hard time finding.
However, at least one trace shows four git for-each-ref executions, which seems suspicious.
Anecdatally, repo001 is under relatively high ongoing load and mostly just observing repositories. I believe the git wire protocol is not especially well-optimized to make no-op fetches as cheap as possible (e.g., all refs are listed on connection) but in theory we should be able to do better than we are.