I did read and take to heart your [[ https://secure.phabricator.com/book/phabricator/article/exit_codes/ | article on exit codes ]]. I am hover faced with a tricky situation in that the exit code used for an unsound test is 1 in ArcanistUnitWorkflow.php.
The issue that we are running into specifically is that we have a harbormaster build rule that runs arc unit. In this rule we rely on the exit status of arc to determine if it passed or failed. In practice this means that a status of 0 or 1 means everything is "fine" and a status of anything else means it failed. however in doing some testing today I noticed that it's also possibly for arc to blow up before even getting to running the unit tests in which case all the unit tests get skipped and we have no way to distinguish this output or status code from that of an unsound test.
```
Exception
2 Command failed with error #1!
3 COMMAND
4 '/var/drydock/workingcopy-3399/repo/super/secret/sauce' -c '/var/drydock/workingcopy-3399/repo/super/secret/sauce/something.yml' --manifest
5
6 STDOUT
7 (empty)
8
9 STDERR
10 Traceback (most recent call last):
11 File "/var/drydock/workingcopy-3399/repo/super/secret/sauce", line 724, in <module>
12 sys.exit(main())
13 File "/var/drydock/workingcopy-3399/repo/super/secret/sauce/run_unit_tests", line 694, in main
14 print json.dumps(generate_manifest(), indent=2)
15 File "/var/drydock/workingcopy-3399/repo/super/secret/sauce/run_unit_tests", line 635, in generate_manifest
16 for test in discoverTests():
17 File "/var/drydock/workingcopy-3399/repo/super/secret/sauce/run_unit_tests", line 276, in discoverTests
18 for filename in sorted(os.listdir(unitTestDir)):
19 OSError: [Errno 2] No such file or directory: '/var/drydock/workingcopy-3399/repo/super/secret/sauce/bin/tests'
20
21 (Run with `--trace` for a full exception trace.)
```
It seems like in this case error codes are already being used to signal the status of the command and having unsound overlap with a catastrophic failure could be problematic.