I feel like I've talked about this before, but I can't find an issue referencing it so I'm creating a new one.
We've turned off pygments on our local phab install because it's too CPU intensive to use, mostly because it spawns a new process for every file it has to colorize.
While I haven't done the work to test it, I'm guessing that pygments would be performant (enough) if we could use it as a server. And today I ran across such a server:
Would there be interest in adding support of this to phabricator? I'm thinking it would be a config option to use the server rather than the binary (and you'd perhaps configure it with the port to talk to), and if you use that config option it's your responsibility to make sure that the server is running on the phabricator machine.
While I don't know how phab uses pygments, I'm hoping it would be a pretty straightforward change that just replaces a system() call with an http request.