Oh well the original FB post was talking about autoscrolling. Anyway I cannot really repro the regular mouse wheel scrolling scenario.

Our HEAD is 4a6d2e9 (D8792) but we do have the fix for T4987 in. I am not quite sure but I think the issue here is that when you are holding the mouse wheel and then move the mouse to scroll up/down (only supported by FF I guess). For other files in https://secure.phabricator.com/D9871 it works all fine, except resources/sql/autopatches/20140710.workerpriority.sql. Although there is no horizontal scroll bar showing up, somehow it is still preventing scrolling up/down if you start trying this scroll mode there.

Ah yes. This could be the one. Thanks Evan!

We actually have it.

Oh damn thank you Evan!

I am at the 4/16 version. I think the hash is 4a6d2e9

One more question on the task daemon behavior: when a task failed (like herald worker) and the daemon restart after a short period, would it retry the failed task immediately since the lease is not expired yet? I read the code and my understanding is that we always get the unleased tasks first, but in production it seems to be failing repeatedly. I am confused on that a little bit.

Ah yes it is 4366, sorry.
Both tables have about 400+ GB (data+index). Yeah I'll explore the alternatives you mentioned for the files.
Actually we already have D7885 and D7887 in our instance. I'll try to dig out more detailed information how it is failing.

A few more issues popping up during the past few weeks - most are general issues and not quite related to the rebase, but I put them here for some initial discussion anyway. I'll file individual tasks later on.

1. Data compression. Some tables are taking hundreds of GB nowadays and keep growing. We need to address it since the total DB size already reached 2.2 TB and the hard limit is 2.5TB. Two major ones are the file blob and the differential hunks. We can enable compression in the table level in MySQL, but it's not as efficient as field compression from php code.
2. In configerator repo we have a folder with more than 16k files (all the sitevars). Open it in diffusion will flood and DOS the whole site. Should we limit the requests sent out here? Maybe only the ones that are visible on the screen should have the request out.
3. Recently we had some issues with the herald rules when the commit is huge (a branch cut for example). Basically if the commit rule has a condition with diff content, it will fail there, and after the daemon restarts it will fail there again and so on. We should be able to handle this type of failures better, or maybe we should not try to get the diff content at all if we know it's going to be enormous.
4. Related to 3, the repeatedly failing herald worker will generate some huge emails in each run, and flood the active task queue. All other tasks are essentially blocked. The task queue is roughly a FIFO queue without taking into consideration the task types.

One more complaint from Facebook: could we bring the author name back to blame view?

I've put up a paste P1075 for the custom code we try to apply during data migration for D7139. Please take a look if you have some time. I've tested it on a test tier and it works fine to me.

Yeah you are right we don't need to have special action on D7513 - just batch the DB queries to make the migration faster I think. For D7139, double writing is not that hard I think.
Also thanks for the heads up on new big migrations - I'll look into them later once the current ones are done. Unfortunately I am living in Seattle, so have to communicate with you this way or on IRC.

Hey Evan and Co. - I think it's better to communicate with you guys with our plan on upgrading phabricator, so that you won't see unexpected comments from Facebook. We want to upgrade to the version of Jan 2014 this time, but there are a couple big data migrations we need to take additional care of. So the whole process will have five pushes:

• Move the production to before D7139
• Do the migration part of D7139 live. The original script is taking too long for us - due to the size of our inline comment table and the location of the DB tier. So we are changing the script a little bit to batch the database access to speed it up (we can do it in 3 hours now). In order to do it live, the inline comment storage code is changed a little bit so it does double writing to both tables whenever inline comments are added/edited/deleted.
• Remove the double writing code in last step, and apply all the code changes up to D7513
• Do the migration part of D7513 live. Again we do similar things - batch database access, do double writing
• Remove the double writing code in last step, and apply all the code changes to some point in Jan 2014 (somewhere around Jan 13 as I remember)

Like this:

The z shortcut is fixed - we moved to the 10/19 version since there is a big migration we need to take special care of, and I picked the fix in for 'z' later yesterday.
The biggest complaint so far on UI is the big white header in diffusion - it seems to be some waste space with just a few menu items which are seldom used.