Change Details

There are several use cases for allowing `arc` to perform more complex "build-like" logic on various workflows. A few examples are: - (T9109, T9223) Run a single build program (like `gradle`) before `arc diff`, then read both lint and unit results from the output. - Run code generation steps before `arc diff`, and after merging with `arc land` or `arc patch` (for example: the Phabricator upstream has generated maps which would be easier to deal with if they were regenerated automatically). - Ask some external system for permission during `arc land`, and prompt the user about inclement weather (for example: tests are failing at HEAD, the branch is considered retired, or release engineering has frozen it). Although we could pursue solutions to this class of problems individually in a tailored way, I think requirements in this class have complexity which exceeds what we can reasonably do by inventing JSON DSLs and adding hard-coded extension points. The `gradle` stuff needs to coordinate with later steps, the "build" and "can I land" stuff would benefit from being independently executable, and various reasonable workflows in this class have sophisticated processing, working copy mutation, or user interaction requirements. I think it's reasonable for us to push `arc` in this direction, but we need to make it easy to build, distribute, and maintain extensions first. So the pathway forward is: - (T5055) Build a packaging system for Arcanist/Phabricator so this stuff can reasonably be distributed and kept up to date. - Design and implement "internal/build" workflows in Arcanist for extending and interacting with its default workflows. These are both systems with large scope, although we can build a simple version first in both cases and get some useful functionality out of it. (The description of this as a "build engine" makes it sound enormously complex, but the distinction I'm really making is that I think these workflows are too complex to express with JSON DSLs, and plugins which can express them are too hard to manage until we build T5055. The primary motivation for T5055 is currently linter / unit test plugins, but once it lands it also gives us much better tools to explore extending other workflows.)

There are several use cases for allowing `arc` to perform more complex "build-like" logic on various workflows. A few examples are: - (T9109, T9223) Run a single build program (like `gradle`) before `arc diff`, then read both lint and unit results from the output. - Run code generation steps before `arc diff`, and after merging with `arc land` or `arc patch` (for example: the Phabricator upstream has generated maps which would be easier to deal with if they were regenerated automatically). - Ask some external system for permission during `arc land`, and prompt the user about inclement weather (for example: tests are failing at HEAD, the branch is considered retired, or release engineering has frozen it). - On a pure Harbormaster/Drydock build pipeline, things are much simpler if `arc unit` can know how to perform a build and `arc unit --everything` can just work. Although we could pursue solutions to this class of problems individually in a tailored way, I think requirements in this class have complexity which exceeds what we can reasonably do by inventing JSON DSLs and adding hard-coded extension points. The `gradle` stuff needs to coordinate with later steps, the "build" and "can I land" stuff would benefit from being independently executable, and various reasonable workflows in this class have sophisticated processing, working copy mutation, or user interaction requirements. I think it's reasonable for us to push `arc` in this direction, but we need to make it easy to build, distribute, and maintain extensions first. So the pathway forward is: - (T5055) Build a packaging system for Arcanist/Phabricator so this stuff can reasonably be distributed and kept up to date. - Design and implement "internal/build" workflows in Arcanist for extending and interacting with its default workflows. These are both systems with large scope, although we can build a simple version first in both cases and get some useful functionality out of it. (The description of this as a "build engine" makes it sound enormously complex, but the distinction I'm really making is that I think these workflows are too complex to express with JSON DSLs, and plugins which can express them are too hard to manage until we build T5055. The primary motivation for T5055 is currently linter / unit test plugins, but once it lands it also gives us much better tools to explore extending other workflows.)

There are several use cases for allowing `arc` to perform more complex "build-like" logic on various workflows. A few examples are: - (T9109, T9223) Run a single build program (like `gradle`) before `arc diff`, then read both lint and unit results from the output. - Run code generation steps before `arc diff`, and after merging with `arc land` or `arc patch` (for example: the Phabricator upstream has generated maps which would be easier to deal with if they were regenerated automatically). - Ask some external system for permission during `arc land`, and prompt the user about inclement weather (for example: tests are failing at HEAD, the branch is considered retired, or release engineering has frozen it). - On a pure Harbormaster/Drydock build pipeline, things are much simpler if `arc unit` can know how to perform a build and `arc unit --everything` can just work. Although we could pursue solutions to this class of problems individually in a tailored way, I think requirements in this class have complexity which exceeds what we can reasonably do by inventing JSON DSLs and adding hard-coded extension points. The `gradle` stuff needs to coordinate with later steps, the "build" and "can I land" stuff would benefit from being independently executable, and various reasonable workflows in this class have sophisticated processing, working copy mutation, or user interaction requirements. I think it's reasonable for us to push `arc` in this direction, but we need to make it easy to build, distribute, and maintain extensions first. So the pathway forward is: - (T5055) Build a packaging system for Arcanist/Phabricator so this stuff can reasonably be distributed and kept up to date. - Design and implement "internal/build" workflows in Arcanist for extending and interacting with its default workflows. These are both systems with large scope, although we can build a simple version first in both cases and get some useful functionality out of it. (The description of this as a "build engine" makes it sound enormously complex, but the distinction I'm really making is that I think these workflows are too complex to express with JSON DSLs, and plugins which can express them are too hard to manage until we build T5055. The primary motivation for T5055 is currently linter / unit test plugins, but once it lands it also gives us much better tools to explore extending other workflows.)