Split large path lists into blocks when linting
f4aadb96048b
Actions

Description

Split large path lists into blocks when linting

Summary:
Fixes T5097. When linting a large list of paths (e.g., with --everything), do this internally:

$chunks = array_chunk($paths, 32);
foreach ($chunks as $chunk) {
  $this->lintChunk($chunk);
}

This keeps the advantages of parallelism and artifact sharing for future-based linters, without having memory usage grow in an unbounded way.

These callbacks changed:

willLintPath(): Useless, no meaningful implementations. Internalized the required side effect and broke the hook.
didRunLinters(): Now useless, with no meaningful implementations. Broke the hook.
didLintPaths(): New hook which executes opposite willLintPaths().
lintPath(): Linters no longer need to implement this method.

XHPAST now has an explicit way to release shared futures.

These minor changes also happened:

Formalized the "linter ID", which is a semi-durable identifier for the cache.
Removed linter -> exception explicit mapping, which was unused. We now just collect exceptions.
We do the canRun() checks first (and separately) now.
Share more service call profiling code.
Fix an issue where the test harness would use the path on disk, even if configuration set a different path.

Test Plan:

Ran arc lint --everything in arcanist/.
- With no chunking, saw unstable memory usage with a peak at 941 MB.
- With chunk size 32, saw stable memory usage with a peak at 269 MB.
- With chunk size 8, saw stable memory usage with a peak at 180 MB.
Ran with --trace and saw profiling information.
Created this diff.