HomePhabricator

Count lines in build log slices more cheaply

Description

Count lines in build log slices more cheaply

Summary:
See PHI766. Ref T13164. Build log chunk processing does a preg_split() on slices, but this isn't terribly efficient.

We can get the same count more cheaply by just using substr_count() a few times.

(I also tried preg_match_all(), which was between the two in speed.)

Test Plan:

  • Used bin/harbormaster rebuild-log --id X --force to rebuild logs. Verified that the linemap is identical before/after this change.
  • Saw local time for the 18MB log in PHI766 drop from ~1.7s to ~900ms, and preg_split() drop out of the profiler (we're now spending the biggest chunk of time on gzdeflate()).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13164

Differential Revision: https://secure.phabricator.com/D19545

Details

Provenance
epriestleyAuthored on Jul 27 2018, 8:22 PM
epriestleyPushed on Jul 30 2018, 3:25 PM
Reviewer
amckinley
Differential Revision
D19545: Count lines in build log slices more cheaply
Parents
rP690a460c8e8d: Allow mailers to be explicitly marked as inbound or outbound
Branches
Unknown
Tags
Unknown
Tasks
T13164: Plans: 2018 Week 31 - 33 Bonus Content
Build Status
Buildable 20532
Build 27884: Run Core Tests