Page MenuHomePhabricator

Evaluate the viability of virtualizing Git refs in hosted repositories
Open, NormalPublic

Description

To perform change handoff to build systems, it would be ideal to represent diffs as virtual refs in hosted repositories.

  • arc could upload changes by pushing to a virtual ref (like review) instead of by shipping JSON over Conduit.
  • The repository would accept the change and redirect the write to some known internal ref (like phabricator/diff/12345) while making the client believe the write occurred cleanly. It could then lift the change out of the ref and into Differential if the ultimate destination was a diff.
  • These refs would be hidden from the web UI and from git clone, so users wouldn't see them normally or have their workflows gummed up by a lot of internal glue.
  • An alternate view of the repository used by build systems could see and interact with these refs.

For example, the goal is to enable workflows like these:

Create a new diff by pushing (see T5000):

$ git push origin diff
Creating a new diff:

  Diff URI: https://secure.phabricator.com/differential/diff/123/

(Also this should exit with 0 as an exit code.)

Create a new durable ref by pushing to some other ref, then get JSON back:

$ git push origin diff/json
{
  "id": 123,
  "remote-ref": "phabricator/diff/123",
  "blah1": "other information useful for arc",
  "blah2": "like we ran differential.creatediff"
}

Check out a remote and only see meaningful, human-facing refs:

$ git clone ssh://dweller@vault.phabricator.com/diffusion/X/x.git
$ cd x
$ git ls-remote origin
abcdef refs/heads/master

Add an alternate view of a remote and see "virtual" refs which are useful to a build system:

$ git remote add build-view ssh://dweller@vault.phabricator.com/diffusion/build/X/x.git
$ git ls-remote build-view
abcdef refs/heads/master
abcde1 phabricator/diff/123
abcde2 phabricator/diff/124
... <a million more entries> ...

Check out those refs:

$ git fetch build-view diff/123
(This actually works.)
$ git checkout build-view/diff/123
(This also works.)
$ run fancy builds
(This works if your code is in good shape.)

Enforce ref-based permissions checks:

$ git fetch build-view diff/456
You Shall Not Pass: You don't have permission to see this diff.

Users who can see this diff include: ...

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a project: Harbormaster.
epriestley added a subscriber: epriestley.

The most promising way to implement these capabilities is to more fully proxy the Git protocol, as we do to some degree for Mercurial and SVN, so I plan to evaluate that first.

epriestley moved this task from Backlog to v1 on the Harbormaster board.May 6 2015, 3:28 PM
joshma added a subscriber: joshma.Jun 1 2015, 6:19 AM
epriestley moved this task from v1 to Future on the Harbormaster board.Aug 10 2015, 7:12 PM
avivey added a subscriber: avivey.Nov 25 2015, 12:41 AM
eadler added a project: Restricted Project.Aug 5 2016, 4:44 PM
kmrhb added a subscriber: kmrhb.Aug 9 2017, 2:22 PM
mves added a subscriber: mves.Oct 11 2017, 10:04 AM