See PHI1605. An install is encountering a change in behavior in CircleCI hooks, possibly associated with CircleCI dropping the HARBORMASTER_BUILD_TARGET_PHID key from build_parameters. However, the whole flow still works correctly for me locally and I can't find any evidence in the CircleCI announcements that they've changed behavior.
The next diagnostic step I'd like to take is to inspect the actual inbound payload from CircleCI. However, this is currently difficult to do in production because no full payload logging exists (limited logs like the Apache access log are available, but do not have enough information to reveal the value of build_parameters).
There are several similar objects already which could likely benefit from a dedicated HTTP logging abstraction:
- Outbound requests from webhooks.
- (Outbound requests from image fetching, repository fetching, etc.)
- Inbound requests from CircleCI.
- (Inbound requests from BuildKite, mailers, etc.)
All of these could reasonably be represented by a "Logged HTTP Request" object of some sort.
One concern with this approach is that many of these logs can not be safely made available to any user via the web UI. For example, if CircleCI sends us a request with no HARBORMASTER_BUILD_TARGET_PHID (as may be the case here), it is unsafe to show to anyone: it may disclose information they do not have permission to see. This dramatically reduces the utility of doing this.
A simpler approach would be a Phacility-specific preamble injection which just writes to a logfile if the request matches certain parameters (instance name, etc). This is pretty icky but a whole lot less involved.