Page MenuHomePhabricator

Add branch name parameter to Harbormaster "Make HTTP Request" build step
Open, LowPublic

Description

We have starting using Harbor Master to drive Jenkins for both Differential and plain continuous integration. It would be very useful to have another variable buildable.branch (The branch for the commit, if applicable.).

Example uses:

  • being able to do different things in the build script based on building a feature branch vs master
  • properly naming build artifacts like app binaries by including branch name

Event Timeline

swisspol raised the priority of this task from to Needs Triage.
swisspol updated the task description. (Show Details)
swisspol added a subscriber: swisspol.

In our case Differential builds are triggered from Herald and so are CI builds (with a basic rule watching for commits on the repo). I assume the branch info might already be available inside Phabricator so hopefully this is not complicated to implement.

If you can think of a workaround in the meantime, please let me know.

You can probably use the diffusion.resolverefs Conduit call to resolve the commit into a list of branches and tags.

Use it like this:

> echo '{"commit":"6d51958b5c0e","callsign":"P"}' | arc call-conduit diffusion.refsquery
Waiting for JSON parameters on stdin...
{"error":null,"errorMessage":null,"response":[{"ref":"HEAD","href":[]},{"ref":"master","href":[]}]}

and pipe the output to something that can handle the JSON blob.

buildable.branch is actually probably not trivial to implement given that in a DVCS like Git you can have multiple branches pointing to the one commit, and there's a question about how to represent multiple branches in that case.

Good point. So how about buildable.branches which would behave like this:

  • Build triggered from Differential: by definition, there is only 1 branch
  • Build triggered from Diffusion / commit: it's ok to provide all branches and in arbitrary order (it would still let us fix our problem if only by checking if the commit is on the master branch at all)

Where else can a build be triggered from?

The question is how to structure more than one branch. Should it be encoded in JSON (in which case we make it hard for shell scripts to parse) or do we separate it with a character (which makes it non-standard and require custom parsing)? We can't use new lines (the default shell scripting separator), because the variable needs to be able to be passed through on both UNIX and Windows command lines.

If comma or space is not a valid character for branch names in SVN, Git and Mercurial, I would just either one as it's trivial to parse in any language. JSON seems overkilled.

Mercurial branches can have pretty much anything, including newlines.

We also can't currently determine which branches the commit was on when we discovered it, only which branches it is currently on. The "multiple branches" thing is the major issue here, but the value being mutable across runs (and even within the same build) also seems like it could cause problems.

epriestley renamed this task from Add branch name parameter to Harbor Master "Make HTTP Request" build step to Add branch name parameter to Harbormaster "Make HTTP Request" build step.Aug 13 2014, 4:05 PM
epriestley triaged this task as Low priority.
epriestley added a project: Harbormaster.

Upshot:

  • Encoding is a mess, especially with Mercurial's liberal policy on what comprises a valid branch name.
  • Branch mutability is a potential landmine.
  • diffusion.refsquery provides a workaround if the other side of the connection can talk to Conduit. We'd like to make talking to Conduit easier so it can solve more of these problems.
  • We might do this eventually anyway, but probably not until Harbormaster is more stable.

Maybe only git and svn should be supported then?

In any case, how do they do it at GitHub? When connecting to Travis CI, you do get a branch name passed in?

From a quick scan of the documentation, it looks like Travis is branch-oriented. I'd guess you get two separate builds of the same commit if you push a commit to multiple branch heads.

Harbormaster is explicitly not branch-oriented. We may add stronger branch binding later for convenience, but it would probably be an attribute of the plan (e.g., plan.triggering-branch), not of the buildable.

Just a thought -- when a build is trigged by Herald, could the Herald (or some subset) be passed and available to Harbormaster?

In my current setup, Herald triggers a build when "commit's branches" matches /develop/. If this (and similar) information could be available to Harbormaster, users may find that helpful.

Ah, yeah -- that's a good idea, and shouldn't have any of the issues that branches have.

Just a thought -- when a build is trigged by Herald, could the Herald (or some subset) be passed and available to Harbormaster?

In my current setup, Herald triggers a build when "commit's branches" matches /develop/. If this (and similar) information could be available to Harbormaster, users may find that helpful.

I think having a plan setting that is "cares about branch", essentially could solve a lot of the shell problems. If you set that to true, then Harbormaster would trigger a build for each branch on the buildable (like Travis CI), each with only one branch available. If you set that to false, then no branch information would be available and if you actually want to have builds that do something intelligent on multiple-branch-commit, you'd have to implement that client side.

The Herald information, while more generic, will also be harder to parse (albeit not impossible). :)

eadler added a project: Restricted Project.Aug 5 2016, 4:54 PM

Hi~

I'm using phabricator to do continuously integeration. I need to judge which branch is updated when jenkins receives the http request.

It's quite difficult for me to paramaterize all the git repositories in phabricator with just one job if there is no branch specified. Sorry I don't know php quite much, otherwise I will try implement myself. I wish there could be some one to resolve this issue, so that I can make better use with phabricator and jenkins.

Thanks a lot in advance if there is anybody to resolve this.

See Planning for information on timelines and priorities.

I think I've got a small tidbit to add on a specific use-case with Jenkins. I had been using pipeline builds which are working nicely but I just tried to setup a multi-branch pipeline build and this requires the branch name in the Jenkins URL of the HTTP request triggering the build. I suspect in this case the proper thing to do would be to trigger and build twice. I guess that's more of an on-branch-change sort of situation than on-commit and using those triggers in Herald (they look to be present, I haven't used them) and having that information passed on to Harbormaster as mentioned could fill the need.