Change Details

We use a number of different unit testing frameworks across multiple projects and we'd like to have this unit test information reported in our builds. Currently we have an xUnit.NET test parser, but because xhpast isn't provided on Windows, it's a real pain for developers to write new unit test and lint integrations. It turns out however, that all of the tools that we're using can output test summaries in TeamCity format, which is documented at https://confluence.jetbrains.com/display/TCD8/Build+Script+Interaction+with+TeamCity#BuildScriptInteractionwithTeamCity-servMsgsServiceMessages. Interestingly enough, this mechanism that TeamCity provides can actually report on more than just unit test results - you can report build statuses, build progress and do other sorts of weird things (like emitting a message which tells TeamCity to download a file as an artifact while the build is still in progress). I thought of submitting a new unit test runner upstream for Arcanist, but that is currently blocked by T5055. I'm pretty sure there's another task where @epriestley explicitly states that upstream doesn't want to accept any more unit test runners because of the maintenance burden. There's a C# reference implementation for reading these messages at https://github.com/JetBrains/TeamCity.ServiceMessages/blob/v3.0/TeamCity.ServiceMessages/src/Read/ServiceMessageParser.cs. I'm thinking of some mechanism where you can specify a parser for Harbormaster build logs (when you edit the build step). This parser would then be fed the content as the build is logged, and it can respond by updating the build or create new build targets. If this was built as some sort of `HarbormasterLogParser` abstract class, then we could extend it locally without extensive patching, and in addition, we could specify all this parsing logic on the Phabricator server which is running Linux (where xhpast is readily available). This probably replaces T6139, because we can leave it up to the log parsers to highlight messages that they think are important to respond to.

We use a number of different unit testing frameworks across multiple projects and we'd like to have this unit test information reported in our builds. Currently we have an xUnit.NET test parser, but because xhpast isn't provided on Windows, it's a real pain for developers to write new unit test and lint integrations. It turns out however, that all of the tools that we're using can output test summaries in TeamCity format, which is documented at https://confluence.jetbrains.com/display/TCD8/Build+Script+Interaction+with+TeamCity#BuildScriptInteractionwithTeamCity-servMsgsServiceMessages. Interestingly enough, this mechanism that TeamCity provides can actually report on more than just unit test results - you can report build statuses, build progress and do other sorts of weird things (like emitting a message which tells TeamCity to download a file as an artifact while the build is still in progress). I thought of submitting a new unit test runner upstream for Arcanist that parses these unit test messages, but that is currently blocked by T5055. I'm pretty sure there's another task where @epriestley explicitly states that upstream doesn't want to accept any more unit test runners because of the maintenance burden. There's a C# reference implementation for reading these messages at https://github.com/JetBrains/TeamCity.ServiceMessages/blob/v3.0/TeamCity.ServiceMessages/src/Read/ServiceMessageParser.cs. I'm thinking of some mechanism where you can specify a parser for Harbormaster build logs (when you edit the build step). This parser would then be fed the content as the build is logged, and it can respond by updating the build or create new build targets. If this was built as some sort of `HarbormasterLogParser` abstract class, then we could extend it locally without extensive patching, and in addition, we could specify all this parsing logic on the Phabricator server which is running Linux (where xhpast is readily available). This probably replaces T6139, because we can leave it up to the log parsers to highlight messages that they think are important to respond to.