Page MenuHomePhabricator

Obscure error when a UnitEngine returns wrong result type
Closed, ResolvedPublic

Description

A user reported the below:

I ran "arc diff HEAD~1 --trace" and got the following:
Running unit tests...
>>> [26] <exec> $ ./gradlew :core:clean :core:test --rerun-tasks
<<< [26] <exec> 23,751,727 us
[2015-06-29 22:32:56] ERROR 2: array_merge() expects at least 1 parameter, 0 given at [/Users/hannahintern/khan/android/.arcanist_extension/CommandTestEngine/src/CommandTestEngine.php:28]
arcanist(head=master, ref.master=55fbc331004b), java-test-engine(), phutil(head=master, ref.master=ea6830cc8019)
  #0 array_merge()
  #1 call_user_func_array(string, array) called at [<java-test-engine>/src/CommandTestEngine.php:28]
  #2 CommandTestEngine::parseOutput(integer) called at [<java-test-engine>/src/CommandTestEngine.php:10]
  #3 CommandTestEngine::run() called at [<arcanist>/src/wor
kflow/ArcanistUnitWorkflow.php:186]
  #4 ArcanistUnitWorkflow::run() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1342]
  #5 ArcanistDiffWorkflow::runUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1221]
  #6 ArcanistDiffWorkflow::runLintUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:475]
  #7 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:382]
[2015-06-29 22:32:56] ERROR 2: Invalid argument supplied for foreach() at [/Users/hannahintern/khan/devtools/arcanist/src/workflow/ArcanistUnitWorkflow.php:200]
arcanist(head=master, ref.master=55fbc331004b), java-test-engine(), phutil(head=master, ref.master=ea6830cc8019)
  #0 ArcanistUnitWorkflow::run() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1342]
  #1 ArcanistDiffWorkflow::runUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1221]
  #2 ArcanistDiffWorkflow::runLintUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:475]
  #3 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:382]
[2015-06-29 22:32:56] ERROR 2: Invalid argument supplied for foreach() at [/Users/hannahintern/khan/devtools/arcanist/src/workflow/ArcanistUnitWorkflow.php:268]
arcanist(head=master, ref.master=55fbc331004b), java-test-engine(), phutil(head=master, ref.master=ea6830cc8019)
  #0 ArcanistUnitWorkflow::run() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1342]
  #1 ArcanistDiffWorkflow::runUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1221]
  #2 ArcanistDiffWorkflow::runLintUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:475]
  #3 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:382]

[2015-06-29 22:32:56] EXCEPTION: (InvalidArgumentException) Argument 1 passed to mpull() must be of the type array, null given, called in /Users/hannahintern/khan/devtools/arcanist/src/workflow/ArcanistUnitWorkflow.php on line 285 and defined at [<phutil>/src/error/PhutilErrorHandler.php:200]
arcanist(head=master, ref.master=55fbc331004b), java-test-engine(), phutil(head=master, ref.master=ea6830cc8019)
  #0 PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phutil>/src/utils/utils.php:107]
  #1 mpull(NULL, string) called at [<arcanist>/src/workflow/ArcanistUnitWorkflow.php:285]
  #2 ArcanistUnitWorkflow::run() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1342]
  #3 ArcanistDiffWorkflow::runUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:1221]
  #4 ArcanistDiffWorkflow::runLintUnit() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:475]
  #5 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:382]

It looks like problem is that gradlew gave no output whatsoever (neither success nor failure). The command worked when she ran it again, so maybe it was a transient problem? In any case, the output was confusing -- maybe the code could notice when the user-specified test doesn't have expected output, and give a nicer error message in that case.

Revisions and Commits

Restricted Differential Revision

Event Timeline

csilvers raised the priority of this task from to Needs Triage.
csilvers updated the task description. (Show Details)
csilvers added a project: Arcanist.
csilvers added a subscriber: csilvers.

This looks like an error in custom third-party code you've written (CommandTestEngine), not the upstream.

Specifically, I'd guess:

  • CommandTestEngine isn't handling this case correctly, then calls array_merge() on something like null.
  • It returns what it expects to be an array, but isn't.
  • This causes a secondary error in the upstream code.

We could and should enforce this contract more directly by checking that the $results the engine returns are a list of test results, but that would just make the secondary error more specific ("Unit test engine 'CommandTestEngine' returned results in the wrong format from run(): expected a list of 'ArcanistUnitTestResult' objects.").

epriestley added a revision: Restricted Differential Revision.Jun 30 2015, 2:12 PM
epriestley triaged this task as Normal priority.
epriestley renamed this task from Obscure error when a unittest runner gives no output to Obscure error when a UnitEngine returns wrong result type.Jun 30 2015, 2:24 PM
epriestley closed this task as Resolved by committing Restricted Diffusion Commit.Jul 1 2015, 11:48 AM
epriestley added a commit: Restricted Diffusion Commit.