Page MenuHomePhabricator

Add controls to prevent `pygmentize` from saturating system resources
Closed, ResolvedPublic

Description

I've had discussions with @epriestley about this before, but I am opening this as a separate maniphest. Basically, pygmentize is super resource-intensive and we don't have many controls in place to prevent pygmentize from saturating available system resources. Currently, by opening a bunch of (approximately 10) large merge commits in separate tabs I am able to DOS our Phabricator install (subsequent requests will hit a proc_open() error) and saturate CPU usage on a c1.xlarge EC2 instance.

Related Objects

StatusAssignedTask
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Openjoshuaspence
DuplicateNone
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Wontfixepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedbtrahan
Resolvedbtrahan
Resolvedchad
ResolvedNone
Resolvedangie
OpenNone
ResolvedNone
ResolvedNone
OpenNone
OpenNone

Event Timeline

joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added subscribers: joshuaspence, epriestley.

I have about half a diff for this.

Let me see what I can dig up, I think I accidentally declared branch bankruptcy at one point..

I might be able to give this a go.

I've removed the two major sources of needless/expensive highlighting (D12132, D12141). There's a third that will eventually be resolved by T5501 (roughly, if APC settings are whack you'll run setup checks on every page, including setup checks against pygmentize).

At least for now, I don't plan to go further than that.