@frgtn is reporting string overflow errors in ExecFuture when processing some Herald commits. Specifically, what's happening here is probably:
- git diff or similar of the commit is >2GB, the maximum size of a string in PHP.
- The stdout buffer of ExecFuture fails once it reads 2GB of data.
- The timeout on the git diff is excessively huge right now, but there's no guarantee we won't read 2GB of data in an arbitrarily short amount of time.
The immediate fix is probably:
- Put a hard cap just below 2GB on ExecFuture's auto-buffer size and throw + kill when reading more data than that. Processes like ssh-exec already stream output and will not be affected by this. Processes which do not stream output should be streaming it if they expect to need to parse more than 2GB of data, since it won't work anyway.
This creates an issue, in that you can evade Herald by making a change like this:
- Add some evil patch.
- Add a 2GB text file.
Maybe the best fix for this is to add a "Change is too large to process" field. Installs that care could write paranoid rules or blacklist these changes from being pushed by most users.
We could also try to process these changes anyway, but we fundamentally can't regex a string >2GB and probably no one expects us to anyway. We'd have to change all the other Herald APIs to return iterators and streams, too, which seems like a big pain.