• Also fix some unit test behaviors.

This was resolved by D21452. PHI1880 has some unrelated additional plans.

I deployed this and it seems to be working properly.

Hey, it worked once. Good enough for me!

No dice. We need bin/upgrade to run before mysql because it has to mount the data volume. So now I'm trying this:

... service ... start rather than service ... restart ...

...probably tested...

I deployed this everywhere in the Phacility cluster yesterday and things have been quiet, so I'm assuming it worked until evidence arises to the contrary.

With max_connections at 1024, the server above still looks fine after ~14 days:

D21756 effectively makes all Git pathways call setSudoAsDaemon(true).

Just for visibility, the error messages you'll see if you're affected by this issue look something like this:

...maybe this is an actual bug in Phabricator where some pathways are just missing the "sudo" wrapper?

Elsewhere, I compacted Phacility free instances onto a small number of database servers. I attempted to "fill up" these servers, and succeeded, sort of, but maybe too much -- they eventually fall over and need to be rebooted.

This has some rough edges that I'm not going to deal with for now:

• Found a few more.

Will this work will with Phabricator?

GitHub was refusing mirror pushes with a credential issue (see also T12896, vaguely). I'm not entirely sure which side of things changed, but the config was set up in a previous-generation way anyway (through a "bot" account instead of a "Deploy Key") so I updated things. Looks like mirroring is back online now and this change has propagated.

See T12847. All the technical parts of this are now solved except for billing, but since Phacility is winding down I no longer plan to pursue it.

I resolved this in rCORE320b2854.

After T13630:

Only one instance was impacted by this and I just credited them until 2099. I don't currently expect to pursue this.

I no longer expect to pursue this.

• Hosts in the repo class are now build by Piledriver (see T13630), which automatically creates the rbak device entries, so this error isn't likely to occur again.
• I also don't expect to launch any more hosts.

I compacted secure onto new hardware (T13671) and shut down saux001 ("Land Revision") and sbuild001 (Harbormaster remote builds). I think all the remaining work is covered under T13630 (largely, just a handful of large database migrations remain).

I just swapped configs over without merging the LBs, since it wasn't immediately obvious to me what the Application vs Classic state of the world is and swapping was good enough.

The aphlict/notify stuff still needs to be tweaked. I think the snlb + slb setup can be merged into a single slb with "TCP (Secure)" forwarding now.

Databases are moved and secure is out of read-only mode. I'm going to adjust repository configuration, then I should be able to tear down secure001.

I'm going to put secure back into read-only mode now and move the databases to the new host.

I brought up the new host and pointed the slb001 load balancer at it. The database is still on the old host, and the new host doesn't have repositories yet, but the basics seem to be working.

Merging 003 into 001 worked fine with a few expected tricks (e.g., when secure is in read-only mode, you can't push a change to take it out of read-only mode, since pushing is a write). Next up is launching a modern m4.large secure-pool host and then migrating the data.

I'm putting secure into read-only mode now, with the intent of completing steps 1-5 above.

• Fix missing variable initialization.

• Fix another random PHP 8.1 issue.

• Fix a PHP 8.1 issue in application search code.

• Fix a PHP 8.1 check in local disk storage engine code.

• Fix a PHP 8.1 issue in user querying.

• Fix a piece of user initializing code for PHP 8.1.

• Correct behavior in query clause construction.