I've had discussions with @epriestley about this before, but I am opening this as a separate maniphest. Basically, pygmentize is super resource-intensive and we don't have many controls in place to prevent pygmentize from saturating available system resources. Currently, by opening a bunch of (approximately 10) large merge commits in separate tabs I am able to DOS our Phabricator install (subsequent requests will hit a proc_open() error) and saturate CPU usage on a c1.xlarge EC2 instance.
Revisions and Commits
- Mentioned In
- T7559: integration with reverse caching proxies such as squid, varnish
T3626: Detection of language for some file types isn't good
- Mentioned Here
- T5501: Provide a builtin APC / OpCache diagnostics page and improve OpCache support
D12132: Don't highlight very large files by default
D12141: Don't syntax highlight codebase pattern search results
I've removed the two major sources of needless/expensive highlighting (D12132, D12141). There's a third that will eventually be resolved by T5501 (roughly, if APC settings are whack you'll run setup checks on every page, including setup checks against pygmentize).
At least for now, I don't plan to go further than that.