Page MenuHomePhabricator

How do I publish Phabricator events into remote systems?
Closed, ResolvedPublic

Description

After February 2018:


Before February 2018:

Overview

If you have some external system (like a bug tracker) and you want to publish events into it (like update things when revisions close):

  • DO NOT use Herald.
  • Use feed.http-hooks in most cases.
  • Subclass DoorkeeperFeedWorker if you need more power.
  • If your external system is something like a chatroom, polling feed.query using Conduit (while gross) is also reasonable.

Do Not Use Herald For This!

Herald has architectural tradeoffs which make it very bad for publishing information into a remote system. In particular:

  • Herald actions often run in-process during other workflows, and will block users. Making service calls from Herald will slow down Phabricator for everyone.
  • There is no way to handle failure or retry. Making service calls from Herald will be flaky and unreliable, and there will be no way to fix these problems.

Instead, use one of the other methods.

Use feed.http-hooks In Most Cases

You can configure feed.http-hooks to make HTTP requests to some external server when events occur. This is the simplest and most general-purpose way to publish events: have the remote server accept the request, publish to the external system, and then return an HTTP 200.

This won't block (it happens in the daemons), and has a sane error handling / recovery / retry strategy.

The information available in feed.http-hooks is somewhat limited. If you need more information, file a feature request.

The full feed is published, regardless of view policies, see T5726#88706.

Subclass DoorkeeperFeedWorker

The "real" way the upstream does this (in Asana and JIRA) is through Doorkeeper. Doorkeeper is a large, complex system for interfacing with external tools and representing their objects locally. This is the most powerful approach, and the one you should pursue if you want your integration in the upstream, but probably overkill for most use cases.

This won't block (it happens in the daemons) and has sane error behavior. It gives you more power to process events than feed.http-hooks does, and can let you use user credentials to interact with remote systems.

There's no documentation on this. You can look at the Asana and JIRA publishers if you want to go down this route. This is recommended only if you have significant experience with Phabricator.

Poll feed.query

You can make a Conduit call to feed.query every few seconds. This isn't great, but may be a reasonable solution to some problems. You can look at PhabricatorBotFeedNotificationHandler for an example of this.

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added projects: Herald, Feed, Doorkeeper.
epriestley added a subscriber: epriestley.
aadis added a subscriber: aadis.Jun 25 2014, 5:05 PM

I'm hitting the following stacktrace on trying to publish

2014/06/25 17:02:46 [error] 800#0: *28 FastCGI sent in stderr: "PHP message: [2014-06-25 22:32:46] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/400]
400 Bad Request at [<phutil>/src/future/http/BaseHTTPFuture.php:338]
PHP message:   #0 BaseHTTPFuture::parseRawHTTPResponse(string) called at [<phutil>/src/future/http/HTTPSFuture.php:373]
PHP message:   #1 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:40]
PHP message:   #2 Future::resolve() called at [<phutil>/src/future/http/BaseHTTPFuture.php:278]
PHP message:   #3 BaseHTTPFuture::resolvex() called at [<phabricator>/src/applications/notification/client/PhabricatorNotificationClient.php:47]
PHP message:   #4 PhabricatorNotificationClient::postMessage(array) called at [<phabricator>/src/applications/notification/client/PhabricatorNotificationClient.php:34]
PHP message:   #5 PhabricatorNotificationClient::tryToPostMessage(array) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:182]
PHP message:   #6 PhabricatorFeedStoryPublisher::sendNotification(integer) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:115]
PHP message:   #7 PhabricatorFeedStoryPublisher::publish() called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2139]
PHP message:   #8 PhabricatorApplicationTransactionEditor::publishFeedStory(DifferentialRevision, array, array) called at [<phabricator>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:751]
PHP message:   #9 PhabricatorApplicationTransactionEditor::applyTransactions(DifferentialRevision, array) called at [<phabricator>/src/applications/differential/conduit/ConduitAPI_differential_Method.php:144]
PHP message:   #10 ConduitAPI_differential_Method::applyFieldEdit(ConduitAPIRequest, DifferentialRevision, DifferentialDiff, array, string) called at [<phabricator>/src/applications/differential/conduit/ConduitAPI_differential_updaterevision_Method.php:67]
PHP message:   #11 ConduitAPI

That shouldn't be related. Likely, you have notifications enabled (notification.enabled), but are not running the Aphlict server or it isn't accessible.

srijan added a subscriber: srijan.Jul 23 2015, 9:28 AM

The data available from feed.http-hooks or feed.query is somewhat limited:

{
  "class": "PhabricatorApplicationTransactionFeedStory",
  "epoch": 1458686843,
  "authorPHID": "PHID-USER-somethingsomething",
  "chronologicalKey": "6265012289929728254",
  "objectPHID": "PHID-TASK-somephid",
  "text": "nornagon moved T2826: Some task to Backlog on the Some Project workboard."
},

The text field is sort of okay, but doesn't afford filtering events (regex matching..?) or formatting the event any differently than plain text (e.g. I'd like to bold the task ID and username). It'd be great to have a more machine-friendly form of this information.

Awesome! Thanks for the quick reply @epriestley :)

dereckson updated the task description. (Show Details)Aug 1 2016, 12:16 AM

I'm having some issues using the feed.http-hooks with the system I'm interacting with. My endpoint (trac) requires a CSRF token. I assume there aren't plans to add any sort of (even low-level support) for that? That's totally fine, this is kind of an edge case anyway.

I'm probably going to go the route of using feed.query + polling instead. I think it will be easier. Just thought I'd drop a note about my use case here.

dgzzhb added a subscriber: dgzzhb.May 18 2017, 6:49 AM
zhen added a subscriber: zhen.Jun 18 2017, 9:23 AM
epriestley updated the task description. (Show Details)Feb 9 2018, 11:05 PM
epriestley closed this task as Resolved.May 3 2019, 4:57 AM
epriestley claimed this task.

The answer here is now pretty unambiguously "Use Webhooks". feed.http-hooks is formally deprecated, Herald remains a terrible idea, and anyone brave enough to touch Doorkeeper can probably figure things out for themselves.