Page MenuHomePhabricator

As Harbormaster logs are processed, build a sparse map of byte offsets to line numbers
ClosedPublic

Authored by epriestley on Feb 23 2018, 7:12 PM.
Tags
None
Referenced Files
F13157199: D19139.diff
Sun, May 5, 9:48 AM
F13151610: D19139.diff
Sat, May 4, 6:42 PM
Unknown Object (File)
Tue, Apr 30, 11:25 PM
Unknown Object (File)
Sat, Apr 27, 11:19 AM
Unknown Object (File)
Mon, Apr 22, 10:46 PM
Unknown Object (File)
Fri, Apr 19, 7:27 PM
Unknown Object (File)
Sat, Apr 6, 5:25 PM
Unknown Object (File)
Mar 30 2024, 6:01 AM
Subscribers
Restricted Owners Package

Details

Summary

Depends on D19138. Ref T13088. When we want to read the last part of a logfile and show accurate line numbers, we need to be able to get from byte offsets to line numbers somehow.

Our fundamental unit must remain byte offsets, because a test can emit an arbitrarily long line, and we should accommodate it cleanly if a test emits 2GB of the letter "A".

To support going from byte offsets to line numbers, compute a map with periodic line markers throughout the offsets of the file. From here, we can figure out the line numbers for arbitrary positions in the file with only a constant amount of work.

Test Plan

Added unit tests; ran unit tests.

Diff Detail

Repository
rP Phabricator
Branch
hlog10
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 19654
Build 26616: Run Core Tests
Build 26615: arc lint + arc unit

Event Timeline

Owners added a subscriber: Restricted Owners Package.Feb 23 2018, 7:12 PM
This revision was not accepted when it landed; it landed in state Needs Review.Feb 27 2018, 1:57 AM
This revision was automatically updated to reflect the committed changes.