The use case here is for having serverspec tests that run on our Phabricator hosts (independently of our puppet manifests which do run ./bin/storage upgrade. The problem with using JSON is that it really complicates the actual test case, which would end up looking like this:

context command('./bin/storage status --json') do
  # Not real code, I fail at Ruby.
  JSON(command.stdout)['success'] should eq true
end

Instead of:

context command('./bin/storage status') do
  its(:exit_status) { should eq 0 }
end

I probably got carried away with the exit status and would probably be fine with returning 2 (or some other constant integer) if there are unapplied patches.

I probably can't be bothered writing a --json mode so the alternative for me would be this:

context command('./bin/storage status') do
  its(:stdout) { should_not match(/Not Applied/) }
end

I think that's fairly reasonable as a correctness vs ease of use tradeoff (although test the exit code for 0, too, if that's not automatic)?

Generally, I never want to accept patches which make correctness impossible if someone does actually want to do things right. In my view, this generally means that exit codes can only signal total/catastrophic failure and can't be meaningfully distinguishable (broadly, the system may cause commands to exit with codes we consider "meaningful" without reaching the command actually doing the "meaningful" thing).

For example, you can get a 127 exit code if PHP is not installed on the system.
You can get a 126 exit code if you don't have permission to execute PHP.
You can get a 1 exit code if you try to run a script you can't read (php -f unreadable).
You can get a 143 exit code by SIGTERM'ing the process.
You can get a 137 exit code by SIGKILL'ing the process.

As far as I know, there is no way to enumerate all the possible codes or guarantee that they are stable across systems and shells.

Obviously, Linux as a whole works fine overall with exit codes everywhere and my viewpoint here is philosophically unusual since pretty much everyone is fine with the simplifying assumption that exit codes are fully controlled by the command you ran, but this just feels like a big janky mess that I want no part of to me.

In my view, T6996 is a particularly compelling example of exit codes being unsound for handling application-level errors: it is very difficult to run x | y correctly (i.e., checking errors), and impossible to do so portably (you have to use a bunch of bash nonsense) as far as I was able to determine. Exit codes have a place, I just view it as a place very low in the stack which shouldn't be involved in application-level error behavior.