Page MenuHomePhabricator

Add an upstream JSON fragment lexer
ClosedPublic

Authored by epriestley on Oct 16 2015, 2:54 PM.
Tags
None
Referenced Files
F14018099: D14292.diff
Tue, Nov 5, 4:39 AM
F14011115: D14292.diff
Thu, Oct 31, 6:24 PM
F13996622: D14292.id34498.diff
Wed, Oct 23, 10:40 PM
F13992404: D14292.id.diff
Tue, Oct 22, 4:20 PM
F13989624: D14292.id34508.diff
Mon, Oct 21, 9:48 PM
F13972826: D14292.id34496.diff
Thu, Oct 17, 9:26 PM
Unknown Object (File)
Fri, Oct 11, 10:32 PM
Unknown Object (File)
Fri, Oct 11, 10:03 PM
Subscribers
None

Details

Summary

Ref T9566. This seems to be >100x faster than an older version of pygmentize for JSON inputs with a single long line. See discussion in D14273.

Test Plan
  • Verified runtime improvement by comparing scripts/highlight.php on the one-line.json test case from D14273 with the new lexer enabled and commented out. Saw >10x local improvement. My local base runtime is also >10x better than the runtime from D14273. My laptop might be a little faster than whatever that hardware was, but this is probably mostly pygmentize improvements in my most recent version.
  • Added a test case.
  • Used the highlighter to produce reasonable-looking results:

Screen Shot 2015-10-16 at 7.51.42 AM.png (230×350 px, 19 KB)

Diff Detail

Repository
rPHU libphutil
Branch
json1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 8310
Build 9530: Run Core Tests
Build 9529: arc lint + arc unit

Event Timeline

epriestley retitled this revision from to Add an upstream JSON fragment lexer.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added reviewers: gd, chad.

Actually, just fix lint by excluding __tests__/data/, which seems reasonable for all similar cases.

Just exclude every /__tests__/data/ directory from all linters.

chad edited edge metadata.
This revision is now accepted and ready to land.Oct 16 2015, 3:20 PM
gd edited edge metadata.

(I'm going to hold this until the release cut tomorrow morning since some issues might crop up once it gets a little more use and it sounds like "upgrade pygments" fixed most of this anyway.)

This revision was automatically updated to reflect the committed changes.