Page MenuHomePhabricator
Create Task
Maniphest T5662

Daemons are trying to publish to the notifications server
Closed, ResolvedPublic
Actions

Description

I just noticed these errors on our worker host:

[21-Jul-2014 05:03:25 UTC] [2014-07-21 15:03:25] EXCEPTION: (HTTPFutureResponseStatusCURL) [cURL/7] (http://localhost:22281/) <CURLE_COULDNT_CONNECT> The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 7) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html#CURLECOULDNTCONNECT at [<phutil>/src/future/http/HTTPSFuture.php:363]
[21-Jul-2014 05:03:25 UTC]   #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:39]
[21-Jul-2014 05:03:25 UTC]   #1 Future::resolve() called at [<phutil>/src/future/http/BaseHTTPFuture.php:277]
[21-Jul-2014 05:03:25 UTC]   #2 BaseHTTPFuture::resolvex() called at [<phabricator>/src/applications/notification/client/PhabricatorNotificationClient.php:49]
[21-Jul-2014 05:03:25 UTC]   #3 PhabricatorNotificationClient::postMessage(array) called at [<phabricator>/src/applications/notification/client/PhabricatorNotificationClient.php:34]
[21-Jul-2014 05:03:25 UTC]   #4 PhabricatorNotificationClient::tryToPostMessage(array) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:182]
[21-Jul-2014 05:03:25 UTC]   #5 PhabricatorFeedStoryPublisher::sendNotification(integer) called at [<phabricator>/src/applications/feed/PhabricatorFeedStoryPublisher.php:115]
[21-Jul-2014 05:03:25 UTC]   #6 PhabricatorFeedStoryPublisher::publish() called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitHeraldWorker.php:383]
[21-Jul-2014 05:03:25 UTC]   #7 PhabricatorRepositoryCommitHeraldWorker::publishFeedStory(PhabricatorRepository, PhabricatorRepositoryCommit, PhabricatorRepositoryCommitData) called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitHeraldWorker.php:84]
[21-Jul-2014 05:03:25 UTC]   #8 PhabricatorRepositoryCommitHeraldWorker::applyHeraldRules(PhabricatorRepository, PhabricatorRepositoryCommit) called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitHeraldWorker.php:17]
[21-Jul-2014 05:03:25 UTC]   #9 PhabricatorRepositoryCommitHeraldWorker::parseCommit(PhabricatorRepository, PhabricatorRepositoryCommit) called at [<phabricator>/src/applications/repository/worker/PhabricatorRepositoryCommitParserWorker.php:44]
[21-Jul-2014 05:03:25 UTC]   #10 PhabricatorRepositoryCommitParserWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:87]
[21-Jul-2014 05:03:25 UTC]   #11 PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:124]
[21-Jul-2014 05:03:25 UTC]   #12 PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:19]
[21-Jul-2014 05:03:25 UTC]   #13 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:84]
[21-Jul-2014 05:03:25 UTC]   #14 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:111]

We have two hosts in our Phabricator setup: one app server (running the PHP application + NodeJS server for Aphlict) and one worker server (running the daemons). The worker host is trying to curl the NodeJS notifications server (which isn't running on localhost). Am I meant to set notifications.server-uri on the worker host?

I'm not sure if this has always been happening (and I haven't realised) or if this is relatively new...

Event Timeline

joshuaspence created this task.Jul 21 2014, 5:17 AM
joshuaspence assigned this task to epriestley.
joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added projects: Daemons, Aphlict.
joshuaspence added a subscriber: joshuaspence.
joshuaspence renamed this task from Daemons are trying to publish the notifications server to Daemons are trying to publish to the notifications server.Jul 21 2014, 8:41 AM
joshuaspence updated the task description. (Show Details)Jul 21 2014, 8:43 AM
epriestley added a comment.Jul 21 2014, 1:03 PM

This is expected. The daemons publish transactions (like "commit X closes task Y") and attempt to publish notifications for those transactions (so you'll be notified about commits, task closure, etc).

The machine hosting the notification server should accept connections from any webservers and also from any machines running task daemons. In most cases there's only one machine matching that description so "localhost" works fine, but if you have several machines the setup may need to be a bit more complicated.

For example, you'd set up aphlict.yourdomain.com, point it at whichever host runs the notification server, and then set notifications.server-uri to that domain globally.

• ryanlopez310 closed this task as a duplicate.Jul 21 2014, 1:52 PM
This comment was removed by • ryanlopez310.
joshuaspence reopened this task as Open.Jul 21 2014, 2:01 PM

(not a duplicate)

joshuaspence added a comment.Jul 22 2014, 10:22 PM

I think that this is resolved now, I'll confirm later today.

Would it be possible to have a setup check for this? Add a dummy event to the worker queue that attempts to post a notification and flags an issue if it fails?

joshuaspence closed this task as Resolved.Jul 29 2014, 1:21 AM
Phabricator · Built by Phacility