Page MenuHomePhabricator

Add a `bin/bulk export` CLI tool to make debugging and profiling large exports easier
ClosedPublic

Authored by epriestley on Jan 30 2018, 1:54 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Nov 24, 7:38 AM
Unknown Object (File)
Tue, Nov 5, 5:18 AM
Unknown Object (File)
Oct 21 2024, 11:35 PM
Unknown Object (File)
Oct 21 2024, 11:30 PM
Unknown Object (File)
Oct 7 2024, 11:33 AM
Unknown Object (File)
Sep 30 2024, 8:23 PM
Unknown Object (File)
Sep 28 2024, 2:08 PM
Unknown Object (File)
Sep 24 2024, 11:15 AM
Subscribers
None

Details

Summary

Ref T13049. When stuff executes asynchronously on the bulk workflow it can be hard to inspect directly, and/or a pain to test because you have to go through a bunch of steps to run it again.

Make future work here easier by making export triggerable from the CLI. This makes it easy to repeat, inspect with --trace, profile with --xprofile, etc.

Test Plan
  • Ran several invalid commands, got sensible error messages.
  • Ran some valid commands, got exported data.
  • Used --xprofile to look at the profile for a 300MB dump of 100K tasks which took about 40 seconds to export. Nothing jumped out as sketchy to me -- CustomField wrangling is a little slow but most of the time looked like it was being spent legitimately.

Diff Detail

Repository
rP Phabricator
Branch
export1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 19275
Build 26043: Run Core Tests
Build 26042: arc lint + arc unit