Page MenuHomePhabricator

Multiple coverage columns for different types of code coverage
Open, WishlistPublic


At my job we mainly use Python, an untyped language. However, we've recently started introducing PEP 484 / MyPy annotations, a compile-time layer that lets us add optional static typechecking to our codebase. This leads to files that have some statically-typed lines, and some lines that are not statically-typed.

It would be nice to have a second coverage column in our code reviews (in addition to the test coverage column) to track whether or not something is statically typed.

Other potential usecases I can think of for >1 coverage column:

  • A similar use case can be applied to TypeScript, which is another partially-statically-checked environment
  • Splitting up "is this line covered by unit tests or integration tests?"
  • For cross-platform libraries, it would be great to have, for instance, "is this covered in Win32 tests?" or "is this covered in OSX tests?" columns

Some background:

You can sort of get an idea for what the type coverage column would be used for based off this sample:

Screenshot 2016-07-25 16.27.14.png (682×1 px, 141 KB)

(This is a real HTML report generated by MyPy and would be trivial to convert into the N-U-C style that ArcanistUnitTestResult setCoverage() uses).
typed_function is typed because it has a PEP 484-comatible # type: ... comment. It takes in an int, and outputs a boolean.
untyped_function is untyped because it doesn't have the # type: ... comment.
main_fn is straightforward until we reach line 12 - here, since we call into an untyped function, the return type is uncertain; hence 12, 13 and 14 are marked as 'untyped' (red).

The crux of this request is that I would like to be able to display this data in Arcanist in addition to the existing code coverage column, as (poorly) mocked up here:

Screenshot 2016-07-25 16.40.35.png (974×288 px, 19 KB)

(The first column being "is this line covered in a test?," the second column being "is this line completely typechecked?" Colors don't have to be customizable, it's just for readability.)