Page MenuHomePhabricator

Cut Herald rules off at 1GB of diff text
ClosedPublic

Authored by epriestley on Jan 3 2014, 7:18 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Apr 25, 8:05 PM
Unknown Object (File)
Thu, Apr 25, 2:58 AM
Unknown Object (File)
Sun, Apr 21, 8:48 PM
Unknown Object (File)
Sat, Apr 20, 6:16 PM
Unknown Object (File)
Thu, Apr 11, 10:36 AM
Unknown Object (File)
Sat, Apr 6, 6:38 AM
Unknown Object (File)
Sat, Apr 6, 6:38 AM
Unknown Object (File)
Sat, Apr 6, 6:23 AM
Subscribers

Details

Summary

Ref T4276. When a change is larger than 2GB, PHP can not read the entire change into a string, so Herald can not process it.

Additionally, we already have a time limit for practical reasons, but it's huge (probably incorrectly). To deal with these things:

  • Add an optional byte limit to diffusion.rawdiffquery.
  • Make the query with a 1GB limit.
  • Reduce the diff timeout from 15 hours to 15 minutes.
  • Add a "Changeset is enormous" field. This field is true for changes which are too large to process.

This generally makes behaviors more sane:

  • We'll always make progress in Herald in a reasonable amount of time.
  • Installs can write global rules to handle (or reject) these types of changes.
Test Plan

Set limit to 25 bytes instead of 1GB and ran test console on various changes.

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

btrahan added inline comments.
src/applications/herald/storage/HeraldRule.php
20

what do you have against 24? =P

src/applications/herald/storage/HeraldRule.php
20

It's in another diff, I'm just faking my way through dependencies. :P