@btrahan, this isn't super high-pri but we can start by writing a script in, say, scripts/conduit/api.php which you invoke like this:

api.php <user> <method>

It should then read a parameter payload from STDIN (trusting the 'user' as having been already authenticated by SSH), execute the method in question, and write the result to STDOUT. You can possibly look at "arc call-conduit" for some vaguely similar code, but this can just execute the method directly without actually doing a service call.

Once we have that, I have most of the glue in place to hook it up to SSH a little later:

https://github.com/epriestley/sshd-vcs
https://github.com/epriestley/sshd-vcs-glue

I got this to actually work:

$ arc list --conduit-uri=ssh://local.aphront.com:2222/
  Needs Review       D7: asdf
  Needs Review       D8: asdf
  Needs Review       D9: asdf
  Needs Review       D10: asdf
  Needs Review       D11: <<Enter Revision Title>>
  Needs Review       D12: example revision
  ...

...but doing so required a full stack of hacks that probably need more infrastructure behind them.

Do you imagine killing all the various /scripts/* in favor of conduit.query and this sort of thing? Some chat in IRC just made me think about this as I looked in /scripts/* and found nothing for a problem but there was a reasonable Conduit solution already.

I actually looked in /scripts/ first too because I knew I'd done something to solve that problem (for Khan and their 6000 hg repos IIRC). I think the reason I did it as Conduit instead of a script is because of the large number of parameters and the complexity of their types. If it was CLI, you'd end up with like 20 flags and some of them would need to take JSON arrays, which are a pain to escape in a CLI context.

I generally think scripts are good, and want to consolidate more stuff into multi-workflow scripts in bin/. At some point I'd maybe like to have, e.g., bin/repository import to solve this problem, but I'm not sure how many of the ridiculous number of options/flags we can get rid of. I have a branch which moves on simplifying the edit/create/import flow (I want to clean it up before T603 / hosting) but it has a stack of miscellaneous blockers behind it.

Not sure why Phabricator requires sshd-vcs. Installing a custom SSH server is likely to cause all sorts of issues. I'd recommend going down the same path as Gitolite, which just updates .ssh/authorized_keys when users change and uses the command= option to work out who it is based on their public key. This seems like a much more common approach than OpenSSH with patches.

Synchronizing .ssh/authorized_keys files is a much harder problem at scale than patching sshd to do lookups.

(Incidentally, this is nearly ready.)

@epriestley: If/when this is to be supported in arc, do you imagine using php's libssh2, or execing ssh directly?

I'd exec ssh directly. If there's some major advantage to using libssh2 (but what?), I'd possibly use that optionally if it was available on the host.

The only real blocker here is that I want to be able to open up one connection and ship a bunch of commands over it, not do one ssh per method call. ssh <host> conduit <method> already works, we just pay the ssh startup overhead for every connection, which isn't ideal.

... on the host

Well, "on the client" in this case, I suppose.

Just for the records:

I've played around with the ControlMaster (connection sharing) feature of OpenSSH's ssh:

Running
time ssh -o "ControlMaster no" -o "ControlPath /tmp/sshsock-%h-%p-%r" localhost pwd

with no master running takes about 200-300 ms, and with a master (ControlMaster yes, -N, keep it alive with &) it's around 16ms - so that's a possible plan of attack.

I've also tested with echo {} | ssh ... localphabricator conduit user.whoami, and it does appear to work (Dropping the times from ~650 ms to ~200 on my test server.

For comparison, echo {} | arc call-conduit user.whoami (over http) takes me 120ms.

That probably only works using openssh and not on windows (ControlPath is a socket).

2 cents:

In my current build stack (Which runs mostly in Jenkins, and written mostly in Python), I use conduit over ssh to reduce the amount of ops needed (I.e., no need for both api-token and SSH key; I already have ssh key for checking out code, so it's an easy choice).

The only problem I've had so far was wrt to files - If configured to local file storage, the ssh conduit is handled by the underlaying vcs-user, which causes troubles in the storage area. Moving this to S3/mysql solves that issue.

I'm planning to support this in the arc experimental branch, although initial support won't be optimized. We can look at ControlPath and/or adding batching to the protocol layer to improve performance once it works.

Offhand, we need to make these changes:

• There needs to be a way to specify the SSH URI for an install.
• The SSH URI can have a different host than the HTTP URI today, and this seems reasonable to retain.
• If an SSH URI is specified, we still need to know which protocol to use for HTTP: arc browse T123 must know where to go.
  • We could have arc make a call to figure this out: ask the server what its base URI is.
• When calls fail because there's no authentication information, the prompt should be smarter about directing you to add an SSH key or link a token.
• In T11951, one install would like to disable HTTP tokens entirely. If this becomes possible, the install instructions should never advise you to generate an HTTP token if the remote doesn't support them.

Multiple hostname stuff: Today, the SSH hostname of an install and the HTTP hostname do not need to be the same.

We use this in the Phacility cluster: Phacility HTTP instances are at instance.phacility.com, but serve repositories via vault.phacility.com.

The technical reason for this is mostly that ELBs can't accept port 22 traffic -- or couldn't when I set them up. It looks like new VPC-ELBs may be able to accept on 22. We currently use HAProxy to balance SSH traffic and ELBs for HTTP traffic.

We could require that all hostnames be the same and change the cluster to bounce SSH traffic through the ELB to the HAProxy LBs and use new VPC ELBs, or put HAProxy in front, or whatever else. But this is probably more work for us than letting them differ.

There are also some possible arguments for using a different hostname for Conduit CLI SSH access (conduit.whatever.com) that has different DNS-based load balancing behavior than other SSH access.

Generally, supporting multiple hostnames gives installs greater flexibility in how they balance different traffic, and there aren't many hard technical costs to us to provide this support (with HTTP we must be able to generate links in email, but we have no such constraints for SSH).

In T550#202580, @epriestley wrote:

Multiple hostname stuff: Today, the SSH hostname of an install and the HTTP hostname do not need to be the same.

We use this in the Phacility cluster: Phacility HTTP instances are at instance.phacility.com, but serve repositories via vault.phacility.com.

The technical reason for this is mostly that ELBs can't accept port 22 traffic -- or couldn't when I set them up. It looks like new VPC-ELBs may be able to accept on 22. We currently use HAProxy to balance SSH traffic and ELBs for HTTP traffic.

We could require that all hostnames be the same and change the cluster to bounce SSH traffic through the ELB to the HAProxy LBs and use new VPC ELBs, or put HAProxy in front, or whatever else. But this is probably more work for us than letting them differ.

There are also some possible arguments for using a different hostname for Conduit CLI SSH access (conduit.whatever.com) that has different DNS-based load balancing behavior than other SSH access.

DNS SRV records could be a reasonable workaround here, which would allow you to treat the actual SSH hosting as an implementation detail.

In T550#202580, @epriestley wrote:

I'm planning to support this in the arc experimental branch, although initial support won't be optimized. We can look at ControlPath and/or adding batching to the protocol layer to improve performance once it works.

Has such support actually materialized?

I want to use "arc unit" on a CI server to submit build results to Phabricator, and use the existing SSH key to do so.

Any particular challenges I might encounter while attempting to implement this feature?