When importing a super-mega-ultra-huge repository into Phabricator, while the import chugs along, it is incredibly slow in two distinct ways:
# The initial `git push` seemingly hangs at the end of transferring all the objects - but it is not hung, it is simply waiting on `commit_hooks.php` to run over every single commit first before finally accepting the push (because Herald rules can reject a push as well as a regular hook).
# The actual //import// of all the commits after the hooks have finished takes an absurdly long time. At 533,000 commits, a rate of 1000 commits per hour means it will take nearly 3 weeks to finish the repository import.
Note that you *will not* experience this unless you have a //really// big repository. Even GHC, the Glasgow Haskell Compiler, can import in less than 6 hours, and it has ~45k commits and ~400kLOC. In my case, I'm importing the Linux kernel for some changes (533,000 commits, over 15-16mil LOC at this point), so Phabricator can manage the private tree and push to mirrors.
As @epriestley noted on IRC, it would probably substantially improve both of these pain points if a particular repository could have Herald rules disabled, so that the hooks and import scripts can completely skip these queries for every single commit. Of course, Linux is still going to be slow - the git repository is so huge some operations can take noticeable seconds anyway, so some slowdown at this scale is probably unavoidable.
Also, this is a one time occurrence (it's pretty unlikely it'll be this much of a problem later, as a few thousand commits here or there should go through at least within a reasonable timeframe), but it particularly hurts on the initial import. I may actually want Herald rules on later, but I probably don't want them on an original clean import.
I'll also note some of the numbers might be off; in particular my machine is running btrfs which may be particularly slow on this workload. So I may kill it and try again on a new filesystem and report back.
Here are some steps to reproduce:
# Start up a VM, make a brand new Phabricator instance.
# Create a new, hosted repository.
# Push the Linux kernel into the new repository
# Go play a video game, since `git push` will take about 4-5 hours for `commit_hooks.php` to finish.
# After that's done, go ahead and take a vacation for 2-3 weeks. Hawaii is pretty sweet.
# **OPTIONAL** Return to work and realize you now have to `git push` several thousand more changes back in.