Improve UTF8StringTruncator behavior for huge inputs
ClosedPublic
Actions

Authored by epriestley on Oct 26 2015, 2:53 PM.

Details

Reviewers

chad

Maniphest Tasks

T9632: Maximum execution time of 30 seconds exceeded

Commits

rPHUfe7d8c21a808: Improve UTF8StringTruncator behavior for huge inputs

Summary

Fixes T9632. Currently, when you truncate a very big input (like a huge paste) into a very small output (like a snippet of that paste), it can take a long time. The amount of work we do is proportional to the size of the input.

Reorganize some of the UTF8 code so we can do less work, and only examine about as much of the input as we can possibly need to look at in order to generate the desired output.

Test Plan

This code is well-covered by unit tests.
Added a new unit test which ran in ~4s before the change and runs in ~2ms afterward on my machine (2000x).
Created a huge paste, viewed from web UI.

Diff Detail

Repository

rPHU libphutil

Branch

utfx1

Lint

Lint Passed

Unit

Tests Passed

Build Status

Buildable 8415
Build 9672: Run Core Tests
Build 9671: arc lint + arc unit

Event Timeline

epriestley updated this revision to Diff 34606.Oct 26 2015, 2:53 PM

epriestley retitled this revision from to Improve UTF8StringTruncator behavior for huge inputs.

epriestley updated this object.

epriestley edited the test plan for this revision. (Show Details)

epriestley added a reviewer: chad.

epriestley added a task: T9632: Maximum execution time of 30 seconds exceeded.

epriestley added inline comments.Oct 26 2015, 2:57 PM

src/utils/PhutilUTF8StringTruncator.php
123	Adding `$hard_limit` here is the biggest impact, and just says "only look at the beginning of the string since we don't need more than that".
129	Passing `$string_pv` ("string codepoint vector") here instead of `$string` ("raw string") saves us a little work. Previously, this function immediately called `phutil_utf8v()` on the string again to turn it into a vector.

epriestley added inline comments.Oct 26 2015, 3:00 PM

src/utils/utf8.php
747	The changes here are: Expose a lower-level function which takes output from `phutil_utf8v()` so we don't have to call it twice. Avoid the extra `phutil_utf8v()` call if the first character is a combining character (we'll almost never hit this). Avoid the unnecessary `array_values()` calls on a potentially large array. Slightly optimize the common case of ASCII strings. Try to make the function a little clearer overall.