From IRC, the HTTPS server didn't even start up (no port in netstat). So my first guess is that there's some 'error' handler we need to install, and that should tell us what the issue is (bad SSL key / weird Node stuff / who knows what).

The foundation for this now exists, and core features like browsing user profiles, wiki pages, tasks, etc., appear to work properly in read-only mode.

This is probably fairly straightforward, but I'd like to see some evidence that installs would actually benefit from it before pursuing it.

I vaguely remember these headers really not being very simple last time this came up, too, although maybe I'm misremembering and can't immediately find the discussion.

I think the incidence of this is so low (and should continue to fall in the future) that it's not worth bringing upstream.

Did you guys know that Microsoft is offering a FREE upgrade to Windows 10!
[ Schedule Upgrade NOW ] [ YES! ] [ W-o-w ] [ Reserve Your Windows 10 Today ]

This only affects badges awarded before we began tracking who awarded them. Badges is a prototype application, and I don't think it's worthwhile to maintain code to do something nicer in this case, since the number of affected awards is maybe in the realm of 200 worldwide for all time, of which 80 are on this install.

T6806 might be ripe to resolve one way or another, too.

From T10822, we currently choose [Changed Subscribers] as a stronger action than [Merged] for tasks in some cases in email subject lines. This is probably not the correct heuristic, [Merged] is normally a strong action.

(@eadler's retitle of this bug is fine and I agree that [Merged] should be a stronger action than [Changed Subscribers], I'll make a note about this on T8952.)

We'll take this feedback into account when making future changes to workboards.

When a group of transactions makes several changes, we guess at which change is most likely to be interesting and include that change as the action in the subject line.

This is now in production and appears to be working properly.

Thanks for the report! I think D15719 should fix this; it should be available in master later today and stable in about 24 hours.

There's a new status panel in Config → Notification Servers. I'm consolidating cluster status information there.

We could maybe git ls-tree -r -d -- <tree> and then parse that. Still a lot of stuff, but somewhat less than the other approaches -- it only takes about 20ms to generate for phabricator/. If that output more than 5K paths or whatever we could give up and fall back to an unfolded list.

A big chunk of this is finally in production and seems to be working, which I'm thrilled about.

This is happening generally as part of T10751. I'm just going to close this out since I don't think it's useful as a separate task.

The logs themselves are a bug that's been around for a while, but I think they're from a race condition that I believed only caused things to log messages with no ill effects.

It looks like the DELETE statements can not be expressed in a way which doesn't upset the "unsafeness detector", even when explicitly ordered by primary key. I think this is the relevant MySQL issue:

Here's the section that's intended to explain it. This applies to Mercurial, too (in theory) which is why we don't explicitly mention git fetch.

Alternatively it seems like we can switch to MIXED mode and recommend that, but some installs probably won't be able to and I'm not sure what the tradeoffs are.

It should be fixed in HEAD, it just only regenerates once every 24 hours or something I think.

Should be fixed in HEAD of master. Thanks for the report!

The staging repository allows you to git fetch the change to the build server (which is easier and more reliable), instead of using arc patch.

The diagnostically useful information from the trace is that this:

In https://<your.install.com>/config/history/, is there a configuration change of any kind shortly before 17:19 UTC?

For design reference, here's the GitHub version of this mail (each comment arrives in a separate mail):

Why are you renaming columns?

I slightly reduced the amount of header magic, and the notification stuff is now cluster-ready (in theory, at least). I'm planning to get notifications, web nodes, and databases redundant in production today, and maybe repositories/daemons if nothing else comes up. No issues with repo bookkeeping so far but even if it's smooth sailing it needs a fair bit of UI work, and same with the daemons.

rCOREf9b858d57460 gives web nodes sudo access to svnserve when they're proxying SSH connections.

(Per T10799, T6878 is the underlying issue with why they don't import, but it's currently never materially useful to import them so this hasn't been a priority to fix. Empirically, they do not import on this install but builds work fine -- except see also T10667 for a weird special case mostly unique to this install.)

Staging is required (well, not strictly required, but strongly recommended) to setup CI.

Here's how I modified the configuration on this host (secure.phabricator.com) to account for this change.