Thanks.

Actually, "no longer necessary" might be overbroad. If you have some really bizarre setup with Mercurial repositories, --no-discovery might save you some log spew in some situations. But assuming you're all git, it should no longer do anything in any valid configuration.

For git, is that because discovery only happens on the Almanac device on which the repository is bound to or is it because there is some sort of global lock taken for repository updates? I'm curious to learn more bout how this works.

The major motivation for --no-discovery was to avoid this problem:

• You have a PullLocal daemon running on host A.
• You have Taskmaster daemons running on some host other than host A (say, host B).
• The PullLocal on host A pulls, does discovery, finds new commit C, and queues an "import/parse/publish commit C" task.
• The Taskmaster on host B pulls that task out of queue and starts work.
• But! Host B hasn't updated yet (maybe you are not running PullLocal there, or maybe it just hasn't pulled yet), and doesn't have commit C locally. Commands like git show C don't work.

We have already fixed this for git by removing all git ... commands from the daemons. Instead of running git show C, daemons always make a Conduit call like diffusion.do-git-show { commit = C }.

For remote Subversion repositories, we've always run directly against the remote and never been affected.

For hosted Subversion repositories, I believe we still do some local operations (until T10754).

For remote and hosted Mercurial repositories, we retain a local working copy and do some local operations (until T10753).

If you just have Git repositories, either of these approaches is fine:

• (Discouraged) Put unclustered repositories on a "repository + web" host, run daemons somewhere else. Only run PullLocal on the "repository + web" host. The daemon host does not need working copies and should not run PullLocal.
• (Preferred) Clusterize the repositories, then run as many daemons/hosts as you want wherever you feel like it. They'll always do the right thing (or, if they don't, it's a bug -- not intended behavior).

The daemons have held repository locks during update/discovery for a very long time, so two simultaneous PullLocal daemons haven't interfered with each other directly in many years, I believe. The interference has just been indirect (via queueing tasks for the taskmasters which they may not have local resources to actually execute).

The "clusterize, then things just work" is because:

• When repositories are clusterized, PullLocal only updates repositories which are supposed to be hosted on the current machine, per Almanac definitions. So if you run it on some other machine, it just won't do anything ("Declining to update repository X because it isn't available on this device according to Almanac configuration" or similar).
• When repositories are clusterized and a host processes a Conduit diffusion.git.show-some-commit call, it updates the local working copy first if it needs to be updated, so the "commit C isn't here yet" issue is impossible.

The first rule means that commits can't be discovered before a clustered repository has them somewhere, and the second rule means that a Taskmaster will always see a consistent view of whether a commit exists when it makes Conduit calls.

Thanks for the very detailed response.