PHI979 is a specific issue where we could use better instance controls (around worker count) in the short term. An alternative approach is to hard-coded if ($instance === 'X') { ... } for now. admin could also probably use more workers, I think we get saturated by backups occasionally.

The other adjacent work is:

• T13076: Plans: Phacility cluster caching, renaming, and rebalance/compaction
• T13178: Plans: Support Pact Competitive Update
• T13156: Plans: Improve Phacility UI for managing instance managers and cards

I'd like to try to tackle at least some of this and see how far I get. These things are probably straightforward:

• Kick UI back to full-width.
• Make "instance manager" flows clear on member lists.
• Also, cancelled invites are kind of cluttery-ish right now.
• Remove "Upload Files".
• Deprecate TOTP MFA.
• Fix all the staff permissions for instance status so that the "disable instance" / "suspend instance" buttons actually work again.

The status on T13076 is:

• We have an instances.state API method, but instances aren't actually using it yet.
• Swap to it directly, or fallback-for-404 + rename as a first step. This also needed a bunch of cache work. I think I wrote a little bit of this but it may have been lost to time when I swapped laptops.
• Then we have more flexible support for stuff like "worker limit" and are generally closer to doing the rebalance/compaction.

PHI1010 could use more detailed triage around whether we can squeeze anything from transaction.search without hitting a bunch of other messes. This is adjacent to a lot of other work.

Here's a "before" profile, with this task as the object:

https://secure.phabricator.com/xhprof/profile/PHID-FILE-47g3frqfj7vmdcjfaczc/

Couple of observations:

• Query fragment construction is relatively expensive and can likely be cached and reduced in cost.
• This query loads handles by default, which is unusual, and we don't need handles to serve transaction.search.

~o.o~

State of the world on the instance changes:

On the actual instance hosts side of things (services/), I'm not going to try to get more than D20082 (the change from almanac.service.search to instances.state) out this week. That change "feels" really good to me from a quality perspective, but it's pretty core and far-reaching and hard to completely convincingly test locally so I want to deploy it without too much other stuff going on and make sure I didn't miss anything weird. Once that's out, there's a decent bit more cleanup we can do to get rid of the old cache, and we should be within striking distance of instance renames.

On the management UI side of things (instances/), I'll be happy to get D20058 (the change to modular transactions) out this week. I'm planning to do a few more changes on top of that today, but they're largely UI adjustments and mostly to just fill out the test plan for the actual change -- I'm just going through transaction by transaction, fixing any obvious UI stuff, and then trying to hammer on the workflow a bit to catch anything I missed.

If instances.state sticks the worker count change becomes fairly easy (I think?), but I'm realizing that repo001 will probably just fall over if we deploy it in a generic way (e.g., if each instance gets one more worker per 4 users or something, repo001 will suddenly be able to burst up to 900 million workers). For this week I'll just do another manual bump and figure things out from there if it looks like the world is stable.