Page MenuHomePhabricator

Evaluate upstream support for third-party build systems
Closed, ResolvedPublic

Assigned To
Authored By
epriestley
Sep 21 2015, 8:57 PM
Tags
Referenced Files
F1204358: urls.png
Apr 4 2016, 12:04 PM
F1056824: pasted_file
Jan 8 2016, 2:43 AM
F895553: Screen Shot 2015-10-19 at 6.54.25 AM.png
Oct 19 2015, 1:59 PM
F881141: Screen Shot 2015-10-15 at 3.25.21 AM.png
Oct 15 2015, 10:29 AM
F880223: Screen Shot 2015-10-14 at 8.25.12 PM.png
Oct 15 2015, 3:28 AM
F879674: Screen Shot 2015-10-14 at 5.49.48 PM.png
Oct 15 2015, 12:51 AM
Tokens
"Mountain of Wealth" token, awarded by Beltran-rubo."Mountain of Wealth" token, awarded by siepkes."Mountain of Wealth" token, awarded by bitglue."Mountain of Wealth" token, awarded by dpaola2."Doubloon" token, awarded by vlada."Mountain of Wealth" token, awarded by joshuaspence."Mountain of Wealth" token, awarded by tycho.tatitscheff."Haypence" token, awarded by chad.

Description

Summary

Harbormaster now exists in usable form, and there's some interest in hooking it up to more external build systems.

Writing lots of integrations for third-party software is broadly something the upstream is not well equipped for. We have a small team and it is hard for us to test integrations and keep them up to date. Integrations tend to have a higher maintenance cost than other types of features because we need to keep them working as APIs change. Problems with integrations can also be difficult for us to reproduce and impossible to build a reproduction case for. So our general attitude here is cautious, and we are not rushing to integrate with everything we can.

Build Systems We Know About

SystemUser InterestSupport ComplexityUpstreamOther Support
Harbormaster + DrydockA BitVery High Builtin
CircleCISome (T7869)Moderate Until August 31, 2018
BuildkiteCustomer (T12173)Low Builtin
JenkinsRelatively High (T6518)Moderate Uber Plugin
TeamCitySomeModerate Couchmate Plugin
TravisSome (T5508)Unknown
HudsonNone?Unknown
CodeshipNone?Unknown
ChangesNone?Unknown Builtin?
GoCDSome (T12813)Low(?) kszatan Plugin

Harbormaster + Drydock

Ref: T9123
Status: In Development

This is native builds, where Phabricator hosts the entire build itself. The upstream is interested in this so it's definitely getting built. There is some community interest in various parts of it, even if that doesn't fully extend to driving builds in all cases. We expect this to cover more use cases over time and for a greater amount of logic to gradually live in Harbormaster.

CircleCI

Ref: T7869
Status: Supported

CircleCI had upstream support until August 18, 2018. See T13188.

Buildkite

Ref: T12173
Status: Supported

Buildkite has upstream support.

Jenkins

Ref: T6518
Status: Recommend Community Extension

Jenkins appears to be the most popular build platform among Phabricator users by a wide margin. Support is moderately complex but Uber has a plugin for it (maintained by @sectioneight) that seems pretty good. At least some other installs are using this successfully in a production environment.

I don't currently expect to develop dedicated upstream support for Jenkins, since it doesn't look like there's much we can really do on our side, nor like there's much we'd do differently than the existing plugin does. And we don't have expertise with Jenkins, so I suspect any effort we made here would be very similar, just way worse. I'd expect we'll just point users at this plugin more formally and maybe write a build step which provides a nice facade for it on the Phabricator side (or contribute such a facade to the plugin, since this can also live in third-party code).

Travis

Ref: One mention in T5508
Status: Evaluate After v2 / Wait for Interest

Travis is similar to CircleCI (hosted builds + webhooks) and likely faces many of the same forces. It may also make sense to support in the upstream.

We have seen only tangential interest in Travis support. Since I don't like building one-offs I'd probably build it anyway if we build CircleCI support and it doesn't make the problem way harder/different, but I don't currently believe there's significant interest in Travis from Phabricator users. I believe Travis may be more popular with smaller, primarily open source projects.

TeamCity

Ref: None
Status: v1 Completed

TeamCity, the CI created by JetBrains, is heavily used by .NET developers as well as, by some accounts, the second most used CI to Jenkins. A Harbormaster extension (build step) has been created as well as a plugin for TeamCity that accepts requests from Harbormaster with diff information, applies that diff, reports back to Harbormaster on test information and a build pass/fail. It's not currently in any public repo yet, hoping to have it public by v2 as it's still being vetted.

Hudson

Ref: None
Status: Does anyone use this?

Hudson was forked into Jenkins a while ago when Oracle created some drama. I believe literally no one uses Hudson anymore. However, it still seems to be maintained. I suspect any users that do exist are Very Enterprisey.

Codeship

Ref: None
Status: Wait for interest

I think someone said they use this thing once but I can't find any hits on this install. I've seen some ads for it? But no material interest that I'm aware of.

Changes

Ref: None
Status: Has Native Support?

This is a build system from Dropbox which comes with native support for Phabricator, at least to some degree. We haven't seen other installs adopting it or requests for upstream support from Dropbox. It also has some level of native support. So basically wait for developments here.

GoCD

Ref: T12813
Status: Community extension in development

Pipeline scheduling is working.

Related Objects

StatusAssignedTask
DuplicateNone
Resolvedepriestley
Resolvedepriestley
OpenNone
Resolvedepriestley
Resolvedbtrahan
OpenNone
Resolvedhach-que
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
ResolvedNone
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
ResolvedNone
OpenNone
Wontfixepriestley
Resolvedepriestley
Wontfixepriestley
Openepriestley

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

So just an FYI, I'm currently building a TeamCity Phabricator/Harbormaster Plugin that aims to:

  • Receive a message from Harbormaster indicating build kickoff of a particular diff
  • Plugin uses Conduit to apply patch on top of master and build
  • As the build runs, status messages are continually sent back to Harbormaster via harbormaster.sendmessage, reporting specifically on unit test pass/fail
  • After build finishes, send message to Harbormaster on pass/fail of build

The plugin is ~80% complete, just testing on my local instance of TeamCity.

I'm still just working through this with CircleCI support. No real headway yet, turnaround has been ~24+ hours every time so we're still just discussing the problem.

This is a blocker for us so we contacted Circle support. They advised making a feature request so they can see how much demand there is.
Go to Compatibility with Phabricator, log in, and click the Heart to "vote" for it.

If CircleCI can only build branches, we could theoretically add an option to have arc push to branches instead of tags, although this is a lot of work and complexity on our side to get around what seems like a relatively silly limitation in CircleCI, if the limitation actually exists. I can just poke at the API to see if I can figure out how to build tags, but if someone already has it working that's obviously easier.

FWIW this would be useful in repositories where users can push branches (and maybe only to certain regex-matching branch names) and not tags. coolstory

FWIW this would be useful in repositories where users can push branches (and maybe only to certain regex-matching branch names) and not tags.

Is there any technical reason users couldn't be permitted to also push tags matching a regex? Or is the barrier there purely regulatory (you need to appeal to a committee that meets once a month in a locked closet in the basement or whatever)?

Is there any technical reason users couldn't be permitted to also push tags matching a regex? Or is the barrier there purely regulatory (you need to appeal to a committee that meets once a month in a locked closet in the basement or whatever)?

There's no technical hurdle. I've worked in organizations where QA or Project Management own source repositories and builds and engineering runs its own tools; I'm reflecting that experience in my post. (Of course the theory that controlling the source controls the quality is pretty easy to knock down.) In that sort of situation exporting directly from a Phabricator-owned repository clone seems like the best story anyway.

Yeah -- that make sense, I'm just want to be mindful of how much complexity we're adding to the upstream in order to subvert barriers that have no technical reason to exist.

For example, Facebook once deployed Phabricator with half the machines in Oregon and half the machines in North Carolina, with an artificial 5 minute replication delay between them (see T1969). We declined to add upstream support to address this scenario.

I'm fully onboard with subverting these barriers and this is the primary means by which I accomplished useful work at Facebook, I just don't want this upstream to bear the costs now that we no longer need to. If a feature is useful on its own and subverts barriers, that's great. If it only subverts barriers, that's a harder sell.

Hey,

If anyone's following along here I figured out circleci support for phabricator and posted some of the details at https://discuss.circleci.com/t/compatibility-with-phabricator/183/8 A big issue for us, that I think could be an issue for other people, is that our phabricator instance is firewalled from the rest of the world so while phabricator can post out, circle cannot post in. To resolve this, I used AWS SQS as a message proxy in the middle.

When you guys think about integrating third party build systems, try to think about best ways people can approach it if their phabricator is behind a VPN from the external build system.

Now that I've ran with this a bit, the next cleanup for us is

  1. Use harbormaster build hooks to trigger the build rather than github messages on phabricator diffs
  2. Use a staging area repo that isn't the same one people are committing to because people are complaining about the temporary phabricator tags.

Is there any documentation on configuring Jenkins for commit builds (instead of differential builds)?

Currently we are trying to migrate from Drydock-hosted builds to Jenkins, and the Jenkins plugin provided by Uber only works for differential reviews. As we don't know Java, we can't patch this, and none of the other HTTP plugins I've tried actually run as a post-build step, so there's no way to make a HTTP request when the build fails.

I have a branch up that allows for non-differential builds to submit
harbormaster from Jenkins. I dropped it before the holidays since there
didn't seem to be much interest but can resurrect it if people are
interested:

https://github.com/uber/phabricator-jenkins-plugin/issues/108

Right now we're working around it by having another build plan called "Post results to Phabricator" which has phid and state parameters. This build plan then runs the following PowerShell on a Windows build agent to post the result:

Invoke-WebRequest -Uri https://.../api/harbormaster.sendmessage -Method Post -Body @{"api.token"="...";"buildTargetPHID"=$ENV:phid;"type"=$ENV:state}

Then we configure other build plans to use a trigger another build with parameters post-build step that looks like this:

pasted_file (866×1 px, 70 KB)

Haven't been able to test if this works yet though because I need to upgrade the Phabricator AMI to support SHA2 SSL.

Not super-familiar with PowerShell. If the issue linked above would be helpful to you, could you please add your requirements there and we can coordinate getting it in the Uber plugin? (I'm the maintainer)

This ticket has a ton of subscribers so I'd like to minimize noise.

I would also express interest in Thoughtwork's go.cd: https://www.go.cd/

I know I am not entirely alone, cause this issue exists: https://github.com/gocd/gocd/issues/1576

According to CircleCI this really truly works now, so I'll see if I can push our integration forward shortly.

It seems that tag builds do not support build_parameters. We need to pass parameters, so this is blocked on them again.

https://discuss.circleci.com/t/compatibility-with-phabricator/183/11

I've updated D14284 + D14286 + D14288 for the new API and rebased them onto modern code.

CircleCI fixed build_parameters in their API so support is now in HEAD of master. There's probably some remaining work but it seems to work properly locally. Let us know if you run into issues with it.

@epriestley Configuring a test project with Phabricator and CircleCI, cannot understand what format should the staging area be in.

Tried bare GitHub URL of the project, tried adding /tag/<tag-name>, release/tag/<tag-name>— getting a {"message":"Tag not found"} in all cases.

What kind of format should the staging area URI be in?

Thanks.

The staging area is a Git remote that we'll push to, so it should be a valid target for git push <url> ....

For a GitHub repository, you can find the repository URLs on the main page in the textarea above the directory list:

urls.png (218×1 px, 40 KB)

Either the HTTPS or SSH URLs should work, although one may be preferable depending on how you normally authenticate in your environment. For example, for this repository:

https://github.com/epriestley/poems

...these are valid staging area URLs:

https://github.com/epriestley/poems.git
git@github.com:epriestley/poems.git

An alternate approach may be to run git remote show origin and then copy the Push URL. This will usually be the same as above and correct, although in rare cases it may not be.

(See also some discussion in T8238.)

eadler added a project: Restricted Project.Aug 5 2016, 4:54 PM

While Changes doesn't list any caveats in the readme for https://github.com/dropbox/phabricator-changes, the readme for Changes itself seems to indicate that it's dead:

NOTICE: THIS REPO IS NO LONGER UPDATED

The related repos indicate that Dropbox no longer intends to provide this CI tool for public consumption, so it's probably worth removing it from consideration completely. Its components started being deprecated months ago: https://github.com/dropbox/changes-mesos-framework/commit/cbb2351d45b4231286a18e70e5fea039b121d0a4

So, does this mean that of the list of potentially-supportable CI solutions for use with Harbormaster today, Jenkins is the only available self-hosted open source option?

kszatan updated the task description. (Show Details)

Looks like CircleCI will stop supporting the V1 API, which we use, in about 5 weeks: https://circleci.com/sunset1-0/

epriestley claimed this task.

I generally don't plan to upstream any more support for third-party build tools, since I don't think a future where Phabricator is free glue for a bunch of paid systems is desirable. The pathways forward here are either:

  • Use Drydock/Harbormaster.
  • Via T13229, license integrations from us for paid build systems.
  • Via T5055, write your own glue or ask the company you're paying for builds/build support write it.