In hindsight, welcome.html was a bad move and never saw significant use (at least on our installs). Once dashboards came out, we just installed a dashboard for every user and never went back.

In this case I think it was just me configuring different UI headers to different tiers.

Right now I'm not sure there's any distinction between audits triggered by global rules, and audits triggered by personal rules. That means if anyone sets up a personal Herald rule to trigger an audit, it will cause Diffusion to only show "Partially Audited" (if that person doesn't respond). This is the case on this Phabricator install, where every commit without a review is affected by a personal rule that someone has set up.

Ah actually you're right; the configurations are different between hosts in the local config because different tiers have different configurations.

I've managed to get PhabricatorRepositoryManagementLookupUsersWorkflow using ConduitCall by implementing host identification.

I attempted to start converting PhabricatorRepositoryManagementLookupUsersWorkflow over to use ConduitCall, but there's no way for it to make a call as the omnipotent user at this time, until "Support Host Identity and Authentication" from T4209 gets done. This almost certainly affects conversion of the commit parsing workers as well.

Also, none of these HA tasks are "this is breaking some production or real-world system"; it's just me documenting what does and doesn't work so there's a clear picture on outstanding work for HA to be usable in production systems (or for Phacility SaSS).

(I can give you explicit instructions on setting up a setup like this, and provide you the configuration files, if you want to get a similar setup going)

Yeah I figured that "high availability" was probably not going to be a huge amount of information.

I think public accessibility implying canCDN is the easiest fix here.

Adding the index to the build log table seems to have resolved the performance issues with viewing build logs, so I'm moving this down in priority.

This was caused by Docker killing off the SETGID bit on postqueue / postdrop executables. See https://gist.github.com/porjo/35ea98cb64553c0c718a.

This is the backtrace on an actual child PID:

Backtrace from daemons:

Closed by commit rPf015cb50fe6a.

Can you verify that D10383 fixes the issue?

Oh, that is not supposed to be creating new logs for each entry. It's supposed to add them to the existing log.

Actually we might have to do this per-lease or per-connection, given that if taskmaster daemons are running across multiple machines, then the path to a known_hosts file won't be valid across all of them.

In T5968#33, @epriestley wrote:

where the latter seems to be much more important?

No users have hit issues with the code being different that I've seen, but quite a few have hit issues with config being different.

I think a primary point of confusion is that this is measuring configuration state and not "the code is different", where the latter seems to be much more important? (I read the error message as something to do with code, because it says "out of date configuration" and not "mismatched configuration")

The fix for this is in D10336 and is pending review.

✘ Merged into T5741.

◀ Merged tasks: T5974.

I'll just merge this into the other one.

There are 14 queued tasks; none of them have been processed since I originally made the task. The MetaMTA tasks are now sitting on an expires of -39245.

(Feel free to re-open this task if you want to make it a catch-all for out-of-date daemon stuff, but this issue is at least resolved for me)

This setup issue seems to have disappeared now. I think this was caused by the Docker instance not doing bin/phd stop to shutdown daemons (it just TERMs then KILLs all processes before shutdown) and hence for 6 minutes Phabricator still thinks those out-of-date daemons are still running.

This is what the leased tasks show as well:

I ran bin/mail send-test and then bin/mail list-outbound almost immediately after; this is what I saw:

It does, but the revision is already in a closed status, so it doesn't count when performing the rule (or at least it didn't when we started using it). The exact sequence is something like this:

It'd be nice if this could detect based on the commit content (hashing it), rather than explicitly only allowing one commit per accepted revision.

Closed by commit rPa2a0f002f069.

I've attached a fix for @epriestley to review when he's online. If you could verify that it fixes the issue for your install in the mean time, that'd be appreciated.

If you want to show more information in Harbormaster, you can probably implement a custom build step that starts a build on Jenkins, polls for the logs, and outputs the Jenkins logs into Harbormaster logs. You can also output "link" artifacts when implementing custom build steps, which can link to whatever external URIs you want.

Personally, I'm not a fan of this. If your unit tests fail, then surely you aren't going to deploy code with failed unit tests? Or if you are, then the unit tests obviously don't impact customers / clients / anyone etc in which case they're not affecting the outcome of your build (i.e. what you do does not change based on whether the unit tests pass or fail).

Closed by commit rP6f85c22faffd.

I hit this same issue when trying to use PhabricatorGlobalLock inside Drydock, which appeared to only occur with lots of frequent requests to Drydock. I ended up using beginReadLocking instead, but obviously this is a problem that impacts existing code as well.