(As a side note, the way we'd support this on our ubuntu phabricator machine is to write an upstart script to start the pygments server on startup -- it would also restart the server if it died -- and to run the server under pypy for even more performance happiness.)

The only way forward on Pygments performance with upstream support is porting language lexers to PHP. Writing, deploying, and supporting an additional service is an enormous amount of work for us and not a solution I'm interested in pursuing for this problem: it burdens us with a large ongoing support cost instead of very small mostly-one-time lexer porting costs.

Some possible ways forward:

• If writing, deploying, and maintaining this server doensn't seem like much work to you (and it may reasonably not be), write this server yourself and replace the pygmentize binary on your system with a lightweight client written in a language with a fast startup time that just calls through to the server.
• Or, run the server and instead of writing a client, modify PhutilPygmentsSyntaxHighlighter yourself to use HTTPSFuture instead of ExecFuture and maintain a small local patch (I think this is probably about 10-20 lines of changes).
• You can purchase a support contract and request support for language lexers in PHP, then we'll port them for you.

Writing, deploying, and supporting an additional service is an enormous amount of work for us and not a solution I'm interested in pursuing for this problem

Totally understood, which is why I framed the proposal the way I did: 1) the server is already written, 2) this alternative is only activated via a config option, and 3) the design of the config option makes it clear (I hope) that the pygments server is outside the phabricator world and thus the client is responsible for maintaining it. Of course, there still could be a support cost associated with it. I don't know if phabricator has -- or wants -- an idea of an "unsupported" configuration option.

replace the pygmentize binary on your system with a lightweight client written in a language with a fast startup time that just calls through to the server
modify PhutilPygmentsSyntaxHighlighter yourself to use HTTPSFuture instead of ExecFuture

An intermediate option is to modify the call to ExecFuture to call curl directly, instead of going through a pygmentize wrapper.

It's very possible we'll either modify PhutilPygmentsSyntaxHighlighter as you suggest, or port a few languages to php ourselves. It sounds like you'd be interested in us forwarding the patch to you guys in the latter case, but not the former?

Yeah -- we'll take the patch in the latter case (porting lexers to PHP) but may not have time to review them for a while. We aren't interested in any version of options/support for using HTTP to talk to Pygments in the upstream.

I'd be open to properly modularizing PhutilDefaultSyntaxHighlighterEngine->getHighlightFuture() to use modern modular/extension patterns so a third-party library could provide a HTTPBasedPygmentsLexer or similar, but only in the context of a support contract.

If writing, deploying, and maintaining this server doensn't seem like much work to you (and it may reasonably not be), write this server yourself and replace the pygmentize binary on your system with a lightweight client written in a language with a fast startup time that just calls through to the server.

This is just FYI, but I implemented such a setup. It took about half a day, so not too bad.

The result is at https://github.com/Khan/pygments-server. The README has instructions for getting it to work with phabricator.

If you like, you could point people toward this, with an explicit mention that there is no Phacility support for it (they are welcome to file bugs/questions on the github issue page). Or not. Up to you.

Perhaps another approach that could be considered, doing the colorizing in the browser using something like highlight.js (https://highlightjs.org/). This, of course, puts the CPU burden on the end user instead of the server.