Page MenuHomePhabricator

Working with harbormaster.sendmessage
Closed, ResolvedPublic

Description

Our team is still using 'postponed' results for unit tests, I've been reading through T9134 and realized that the only (sane) way to interact with phab from jenkins is via harbormaster.

  1. When arc diff happens, and we would like to run a set of jobs on jenkins based on the file-types that are changed as a part of that revision, so we could use a herald rule to kick off a harbormaster build plan and listen for progress/pass/fail message from jenkins -- Issue I have here is, how do I pass the knowledge about which jobs to run based on the files changed in the revision -- is there a way to pass a variable from herald to harbormaster?
  1. If, in fact there is no way to send variables from herald to harbormaster, Here is what I'm planning on doing:
    • As soon as an arc diff happens, I would store metadata to the diff using differential.setdiffproperty which include info about the specific jobs needed to be run based on the files changed for that revision and use just one herald rule for the whole repository to call a build plan on harbormaster, which in turn calls a 'master' job on jenkins, which queries for that diff's properties and kicks off downstream jobs based on the metadata passed.
    • Does that sound like a good approach? I would love to hear if there is a better/simpler approach to this problem.
  1. I have been trying to use harbormaster.sendmessage to interact with a test buildable I created via arc diff, following this format
echo '{
  "buildTargetPHID": "PHID-HMBT-7rarhpdxgwgoqkdcptah",
  "type": "fail",
  "unit": [
    {
      "name": "PassingTest",
      "result": "pass"
    }
  ]
}' | arc call-conduit --conduit-uri http://phabricator.ybv.com/ --conduit-token <conduit-token> harbormaster.sendmessage

and the only response i seem to be getting is {"error":null,"errorMessage":null,"response":null}
with --trace

'...
>>> [0] <conduit> user.whoami() <bytes = 117>
>>> [1] <http> http://phabricator.ybv.com/api/user.whoami
<<< [1] <http> 403,749 us
<<< [0] <conduit> 405,462 us
>>> [2] <conduit> harbormaster.sendmessage() <bytes = 292>
>>> [3] <http> http://phabricator.ybv.com/api/harbormaster.sendmessage
<<< [3] <http> 381,401 us
<<< [2] <conduit> 381,642 us
{"error":null,"errorMessage":null,"response":null}

Any idea on what's going on there?

Event Timeline

ybv updated the task description. (Show Details)
ybv added projects: Harbormaster, Herald, Arcanist.
ybv added a subscriber: ybv.

@chad thanks for pointing, I just did, I've looked at the uber plugin before, but for our transition, I wanted to know if any of what I was asking above could be possible before we start entirely relying on that plugin. Also, have you seen that issue with harbormaster.sendmessage previously?

What response are you expecting? The method returns void (meaning "no data") and that's a "success" response (no error occurred).

@epriestley Ah, I see! I thought there was some error that's not being printed. I was expecting something like "message sent". Thanks for clearing that.

@chad @epriestley So, before closing this task, I have a question about how exactly harbormaster listens for messages, I have a build plan that makes a post call to jenkins with some parameters, the response (in the message log) appears to be the head response from jenkins server

Server: nginx/1.2.4
2	Date: Wed, 30 Sep 2015 00:36:58 GMT
3	Content-Length: 0
4	Location: http://10.21.42.131/queue/item/72/
5	Connection: keep-alive
6	X-Content-Type-Options: nosniff
7	Set-Cookie: JSESSIONID.9a9706df=1s6zk8yhj6ggc1wtjv17yqmov0;Path=/;HttpOnly
8	Expires: Thu, 01 Jan 1970 00:00:00 GMT

And the buildable is marked as a failure!
is it because harbormaster didn't receive a sendmessage? or am I doing something wrong?

@chad @epriestley got my setup working with sending messages about the builds via 'work' type result to harbormaster.sendmessage but still seeing the build's status as 'Failed' and the same log..
I'm not running any unit tests locally and no unit.engine is specified in my .arcconfig.. is that the reason?

To summarize,
arc diff -> linting done locally, no unit test engine specified -> herald rule kicking off a harbormaster build plan waiting for message -> jenkins job kicked off by harbormaster (harbormaster seems to mark the build as failure at this point, immediately) -> jenkins job responds with 'type':'work' for the build target -> I can see that in unit test results on the revision.

only problem being the build status for the above stated build plan is marked as 'failure'

I would really really appreciate any help in understanding why the build plan is marked as failure.

We fail if the server returns an HTTP status other than 200, exactly. Since it's returning a "Location" header, it's probably returning a 3xx status alongside it. The behavior you describe is consistent with the HTTP request returning a non-200 status.

@epriestley thanks of the immediate response, looks like jenkins is sending back a 201 Created. I can try to mask it as a 200 but do you think there is value in allowing 2xx status codes on harbormaster's side?

The build step will probably let you configure that eventually, but I don't expect to change that any time in the near future.

ah, i see!
I got too excited and created D14200

no worries! thanks for all the help @epriestley

@epriestley Any jenkins job that has to be built with parameters returns a 201 created, https://issues.jenkins-ci.org/browse/JENKINS-22849
it is turning out to be difficult to make it return 200 to a client even with a reverse proxy in front. Just wanted to know if you'd like to reconsider the decision to support 2xx or at the least, 201?
I wonder how @sectioneight's plugin works around this.

Although the return code check against exactly 200 has always been there, integration with jenkins was working as expected before rPd735c7adf2d5897d3ae08f3aba12774030c0d3b1

After I updated earlier this week, all my builds started failing, but jenkins is sending a "pass" response:

In the Harbormaster Build panel, the following information is shown:

When: Completed at Tue, Sep 29, 8:38 AM · Built instantly
Status: Failed

Messages:

454 jasonrumney pass Tue, Sep 29, 8:44 AM

Even though the Harbormaster task is set to wait for a response, it does not seem to be waiting any more based on those times, while before it was waiting despite the 201 status returned from jenkins.

It seems the change from setting the result to FAIL to throwing an exception in the case where the server does not return 200 is skipping some code after the call that was overriding the result to WAITING in the case where a build was set to wait for result. Probably this was a bug in the case where the result was already set to fail.

An easy fix will be to accept all 200 - 206 return codes, which should be valid as they are all variants on "request accepted and complete".

Aren't 2xx statuses considered "success" in HTTP protocol, despite values of "xx"?

So theoretically code instead of:

 if ($status->getStatusCode() != 200) {
      throw new HarbormasterBuildFailureException();
}

should be:

 if ($status->getStatusCode() < 200 || $status->getStatusCode() >= 300)  {
      throw new HarbormasterBuildFailureException();
}

As of now, I'm hacking around this by having a flask app that listens to harbormaster calls and in turn calls jenkins before returning 200 to harbormaster.

epriestley claimed this task.
  • We now accept any 2xx status as success.
  • For (1) and (2) above, there is currently no support in the upstream, but builds can be parameterized after D14222. See T9352 for discussion and an example of how to write a custom Herald action which does this.