HomePhabricator

(stable) Reduce the impact of `bin/storage dump`

Description

(stable) Reduce the impact of bin/storage dump

Summary:
Ref T12646.

  • Use "wb1" instead of "wb" to use level 1 gzip compression (faster, less compressy). Locally, this went about 2x faster and the output only grew 4% larger.
  • LinesOfALargeExecFuture does a lot of unnecessary string operations, and can boil down to a busy wait. The process is pretty saturated by I/O so this isn't the end of the world, but just use raw ExecFuture with FutureIterator so that we wait in select().
  • Also, nice the process to +19 so we try to give other things CPU.

Test Plan:

  • Ran bin/storage dump --compress --output ....
  • Saw CPU time for my local database drop from ~240s to ~90s, with a 4% larger output. Most of this was adding the 1, but the ExecFuture thing helped a little, too.
  • I'm not sure what a great way to test nice in a local environment is and it's system dependent anyway, but nothing got worse / blew up.
  • Used gzcat | head and gzcat | tail on the result to sanity-check that everything was preserved.

Reviewers: chad, amckinley

Reviewed By: chad

Maniphest Tasks: T12646

Differential Revision: https://secure.phabricator.com/D17795