Page MenuHomePhabricator

XHPAST doesn't build automatically
Open, Needs TriagePublic

Description

I made a trivial change to XHPAST and then ran arc lint:

diff --git i/src/parser/xhpast/bin/PhutilXHPASTBinary.php w/src/parser/xhpast/bin/PhutilXHPASTBinary.php
index ac2dc09..f6fa04e 100644
--- i/src/parser/xhpast/bin/PhutilXHPASTBinary.php
+++ w/src/parser/xhpast/bin/PhutilXHPASTBinary.php
@@ -8,7 +8,7 @@ final class PhutilXHPASTBinary extends Phobject {
    * This is the version that would be obtained with an up-to-date XHPAST
    * build. The //actual// XHPAST build version may vary.
    */
-  const EXPECTED_VERSION = '5.5.8/1j';
+  const EXPECTED_VERSION = '5.5.8/1k';
 
   /**
    * The XHPAST build version.
diff --git i/support/xhpast/xhpast.cpp w/support/xhpast/xhpast.cpp
index cb41f1d..43b98eb 100644
--- i/support/xhpast/xhpast.cpp
+++ w/support/xhpast/xhpast.cpp
@@ -14,7 +14,7 @@ void print_node(xhpast::Node *node);
 int main(int argc, char* argv[]) {
   if (argc != 1) {
     // Coupling: modify also src/parser/xhpast/bin/PhutilXHPASTBinary.php
-    cout << "5.5.8/1j\n";
+    cout << "5.5.8/1k\n";
     return 0;
   }

I get the following exception:

> arc lint --trace
libphutil loaded from '/home/joshua/workspace/github.com/phacility/libphutil/src'.
arcanist loaded from '/home/joshua/workspace/github.com/phacility/arcanist/src'.
Config: Reading user configuration file "/home/joshua/.arcrc"...
Config: Did not find system configuration at "/etc/arcconfig".
Working Copy: Reading .arcconfig from "/home/joshua/workspace/github.com/phacility/libphutil/.arcconfig".
Working Copy: Path "/home/joshua/workspace/github.com/phacility/libphutil" is part of `git` working copy "/home/joshua/workspace/github.com/phacility/libphutil".
Working Copy: Project root is at "/home/joshua/workspace/github.com/phacility/libphutil".
Config: Did not find local configuration at "/home/joshua/workspace/github.com/phacility/libphutil/.git/arc/config".
>>> [0] <exec> $ git rev-parse --verify HEAD^
<<< [0] <exec> 7,530 us
>>> [1] <exec> $ git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}'
<<< [1] <exec> 6,022 us
>>> [2] <exec> $ git cat-file -t 'origin/master'
<<< [2] <exec> 2,835 us
>>> [3] <exec> $ git merge-base 'origin/master' HEAD
<<< [3] <exec> 3,086 us
>>> [4] <exec> $ git diff --no-ext-diff --no-textconv --raw 'c76b692447afe6574a1689491aaf99be0a46b650' --
<<< [4] <exec> 13,173 us
>>> [5] <exec> $ git diff --no-ext-diff --no-textconv --raw 'HEAD' --
>>> [6] <exec> $ git ls-files --others --exclude-standard
<<< [5] <exec> 9,167 us
<<< [6] <exec> 8,573 us
>>> [7] <exec> $ git diff-files --name-only
<<< [7] <exec> 4,302 us
>>> [8] <exec> $ git diff --no-ext-diff --no-textconv --no-color --src-prefix=a/ --dst-prefix=b/ -U32767 -M -C 'c76b692447afe6574a1689491aaf99be0a46b650' --
<<< [8] <exec> 33,502 us
Examining paths for linter 'chmod'.
Found 23 matching paths for linter 'chmod'.
Examining paths for linter 'cppcheck'.
Found 3 matching paths for linter 'cppcheck'.
Examining paths for linter 'filename'.
Found 23 matching paths for linter 'filename'.
Examining paths for linter 'generated'.
Found 23 matching paths for linter 'generated'.
Examining paths for linter 'json'.
Found 0 matching paths for linter 'json'.
Examining paths for linter 'merge-conflict'.
Found 23 matching paths for linter 'merge-conflict'.
Examining paths for linter 'nolint'.
Found 23 matching paths for linter 'nolint'.
Examining paths for linter 'phutil-library'.
Found 2 matching paths for linter 'phutil-library'.
Examining paths for linter 'phutil-xhpast'.
Found 2 matching paths for linter 'phutil-xhpast'.
Examining paths for linter 'spelling'.
Found 23 matching paths for linter 'spelling'.
Examining paths for linter 'text'.
Found 23 matching paths for linter 'text'.
Examining paths for linter 'xhpast'.
Found 2 matching paths for linter 'xhpast'.
>>> [9] <exec> $ '/home/joshua/workspace/github.com/phacility/libphutil/src/parser/xhpast/bin/xhpast' --version
<<< [9] <exec> 11,089 us
>>> [10] <exec> $ which 'cppcheck'
<<< [10] <exec> 5,373 us
>>> [11] <exec> $ which 'cppcheck'
<<< [11] <exec> 5,365 us
>>> [12] <exec> $ 'cppcheck' --version
<<< [12] <exec> 3,979 us
>>> [13] <lint> Generated Code <paths = 23>
<<< [13] <lint> 1,018 us
>>> [14] <lint> Lint Disabler <paths = 21>
<<< [14] <lint> 725 us
>>> [15] <lint> Basic Text Linter <paths = 21>
<<< [15] <lint> 121,395 us
>>> [16] <lint> XHPAST/libphutil Lint <paths = 2>
>>> [17] <exec> $ which 'gmake'
<<< [17] <exec> 3,375 us
>>> [18] <exec> $ 'make' -C '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast' 'clean' 'all' 'install'
<<< [18] <exec> 9,651,182 us
<<< [16] <lint> 9,657,359 us
>>> [19] <lint> Spellchecker <paths = 21>
<<< [19] <lint> 659,978 us
>>> [20] <lint> XHPAST Lint <paths = 2>
>>> [21] <exec> $ which 'gmake'
<<< [21] <exec> 7,594 us
>>> [22] <exec> $ 'make' -C '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast' 'clean' 'all' 'install'
<<< [22] <exec> 9,347,620 us
<<< [20] <lint> 9,360,593 us
>>> [23] <lint> Merge Conflicts <paths = 21>
<<< [23] <lint> 47,604 us
>>> [24] <lint> Cppcheck <paths = 1>
>>> [25] <exec> $ which 'cppcheck'
<<< [25] <exec> 8,413 us
>>> [26] <exec> $ 'cppcheck' '--quiet' '--inline-suppr' '--xml' '--xml-version=2' '-j2' '--enable=performance,style,portability,information' '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast/xhpast.cpp'
<<< [24] <lint> 13,137 us
>>> [27] <lint> Filename <paths = 21>
<<< [27] <lint> 1,496 us
>>> [28] <lint> Chmod <paths = 21>
<<< [28] <lint> 1,040 us
>>> [29] <lint> Phutil Library Linter <paths = 2>
>>> [30] <exec> $ which 'find'
<<< [30] <exec> 3,841 us
>>> [31] <exec> $ find . "(" -path '*/.*' ")" -prune -o -type 'f' "(" -name '*.php' ")" -print0
<<< [31] <exec> 15,480 us
>>> [32] <exec> $ which 'gmake'
<<< [32] <exec> 7,411 us
>>> [33] <exec> $ 'make' -C '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast' 'clean' 'all' 'install'
<<< [33] <exec> 11,852,876 us
<<< [29] <lint> 11,927,269 us
>>> [34] <lint> Generated Code <paths = 23>
<<< [34] <lint> 149 us
>>> [35] <lint> Lint Disabler <paths = 21>
<<< [35] <lint> 128 us
>>> [36] <lint> Basic Text Linter <paths = 21>
<<< [36] <lint> 122 us
>>> [37] <lint> Spellchecker <paths = 21>
<<< [37] <lint> 118 us
>>> [38] <lint> Merge Conflicts <paths = 21>
<<< [38] <lint> 118 us
>>> [39] <lint> Cppcheck <paths = 1>
<<< [26] <exec> 11,940,695 us
<<< [39] <lint> 555 us
>>> [40] <lint> Filename <paths = 21>
<<< [40] <lint> 120 us
>>> [41] <lint> Chmod <paths = 21>
<<< [41] <lint> 123 us

[2015-08-14 00:44:59] EXCEPTION: (PhutilAggregateException) Some linters failed:
    - Exception: xhpast is broken.
    - PhutilProxyException: Your version of 'xhpast' is unbuilt or out of date. Run this script to build it.:
      
        $ /home/joshua/workspace/github.com/phacility/libphutil/scripts/build_xhpast.php
      
    - CommandException: Command failed with error #2!
      COMMAND
      'make' -C '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast' 'clean' 'all' 'install'
      
      STDOUT
      make: Entering directory '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast'
      rm xhpast parser.yacc.output libxhpast.a *.o 2>/dev/null
      Makefile:24: recipe for target 'clean' failed
      g++ -c -fPIC -Wall -O3 -minline-all-stringops -o astnode.o astnode.cpp
      g++ -c -fPIC -Wall -O3 -minline-all-stringops -o scanner.lex.o scanner.lex.cpp
      g++ -c -fPIC -Wall -O3 -minline-all-stringops -o parser.yacc.o parser.yacc.cpp
      ar -crs libxhpast.a astnode.o scanner.lex.o parser.yacc.o
      g++ -fPIC -Wall -O3 -minline-all-stringops -o xhpast xhpast.cpp libxhpast.a
      cp xhpast ../../src/parser/xhpast/bin/xhpast
      Makefile:30: recipe for target 'install' failed
      make: Leaving directory '/home/joshua/workspace/github.com/phacility/libphutil/support/xhpast'
      
      
      STDERR
      make: [clean] Error 1 (ignored)
      cp: cannot create regular file ‘../../src/parser/xhpast/bin/xhpast’: Text file busy
      make: *** [install] Error 1
       at [<arcanist>/src/lint/engine/ArcanistLintEngine.php:273]
arcanist(head=master, ref.master=e1145ef7f812), phutil(head=master, ref.master=7864dbdc3cde)
  #0 <#4> ExecFuture::resolvex() called at [<phutil>/src/future/exec/execx.php:17]
  #1 <#4> execx(string, string, string, array) called at [<phutil>/src/parser/xhpast/bin/PhutilXHPASTBinary.php:43]
  #2 <#4> PhutilXHPASTBinary::build() called at [<phutil>/src/moduleutils/PhutilLibraryMapBuilder.php:446]
  #3 <#4> PhutilLibraryMapBuilder::analyzeLibrary() called at [<phutil>/src/moduleutils/PhutilLibraryMapBuilder.php:96]
  #4 <#4> PhutilLibraryMapBuilder::buildFileSymbolMap() called at [<arcanist>/src/lint/linter/ArcanistPhutilLibraryLinter.php:73]
  #5 <#2> PhutilXHPASTBinary::build() called at [<phutil>/src/parser/xhpast/bin/PhutilXHPASTBinary.php:82]
  #6 <#2> PhutilXHPASTBinary::getParserFuture(string) called at [<arcanist>/src/lint/linter/ArcanistBaseXHPASTLinter.php:112]
  #7 <#2> ArcanistBaseXHPASTLinter::buildSharedFutures(array) called at [<arcanist>/src/lint/linter/ArcanistBaseXHPASTLinter.php:53]
  #8 <#2> ArcanistBaseXHPASTLinter::buildFutures(array) called at [<arcanist>/src/lint/linter/ArcanistFutureLinter.php:17]
  #9 <#2> ArcanistFutureLinter::willLintPaths(array) called at [<arcanist>/src/lint/engine/ArcanistLintEngine.php:580]
  #10 <#2> ArcanistLintEngine::executeLinterOnPaths(ArcanistPhutilXHPASTLinter, array) called at [<arcanist>/src/lint/engine/ArcanistLintEngine.php:541]
  #11 <#2> ArcanistLintEngine::executeLintersOnChunk(array, array) called at [<arcanist>/src/lint/engine/ArcanistLintEngine.php:479]
  #12 <#2> ArcanistLintEngine::executeLinters(array) called at [<arcanist>/src/lint/engine/ArcanistLintEngine.php:217]
  #13 ArcanistLintEngine::run() called at [<arcanist>/src/workflow/ArcanistLintWorkflow.php:336]
  #14 ArcanistLintWorkflow::run() called at [<arcanist>/scripts/arcanist.php:382]

A subsequent run of arc lint works fine.

Event Timeline

joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added projects: Arcanist, XHPAST.
joshuaspence added a subscriber: joshuaspence.

I can sorta fix with using PhutilFileLock, but it feels a bit gross. Here is something I threw together real quick: https://secure.phabricator.com/differential/diff/33545/

I think using FileLock is pretty reasonable. It's a little gross, but I'm not aware of any way to do this synchronization in make itself (although it's possible there is one), and if the process doesn't have write access to the lock it isn't going to be able to run make anyway, so it shouldn't be adding new failure modes.

I'm OK with some flavor of that diff if you want to send it up. Only suggestions are:

  • Maybe wait longer than 20 seconds?
  • Maybe call it phutil.make.lock.lockfile.lockeylock or something idk