arc confused by HTTP->HTTPS redirects
Open, Needs TriagePublic

Description

Behaviour is observable on this very Phabricator instance; I do not know which version that runs. However, it is also observable on the week 29 stable promotions (the instance I run, and first found this on, has a reasonable amount of local changes, but none which touch this codepath, so here are the upstream SHAs):


Have a site which 302 redirects HTTP -> HTTPS, e.g. http://secure.phabricator.com.

Run arc install-certificate http://that.site (not HTTPS!), obtaining a new API token and pasting it in as instructed.

Receive a confusing error message complaining you lack a session key, even though you just got the token from the link like instructed.

strictly:~% arc install-certificate http://secure.phabricator.com --trace
 ARGV  '/home/daniels/src/phabricator/arcanist/bin/../scripts/arcanist.php' 'install-certificate' 'http://secure.phabricator.com' '--trace'
 LOAD  Loaded "phutil" from "/home/daniels/src/phabricator/libphutil/src".
 LOAD  Loaded "arcanist" from "/home/daniels/src/phabricator/arcanist/src".
Config: Reading user configuration file "/home/daniels/.arcrc"...
Config: Did not find system configuration at "/etc/arcconfig".
Working Copy: No candidate locations for .arcconfig from this working directory.
Working Copy: Path "/home/daniels" is not in any working copy.
 CONNECT  Connecting to "http://secure.phabricator.com/api/"...
>>> [0] <conduit> conduit.ping() <bytes = 39>
>>> [1] <http> http://secure.phabricator.com/api/conduit.ping
<<< [1] <http> 1,308,376 us
<<< [0] <conduit> 1,311,549 us
>>> [2] <conduit> conduit.getcapabilities() <bytes = 39>
>>> [3] <http> http://secure.phabricator.com/api/conduit.getcapabilities
<<< [3] <http> 429,877 us
<<< [2] <conduit> 430,641 us
LOGIN TO PHABRICATOR
Open this page in your browser and login to Phabricator if necessary:

http://secure.phabricator.com/conduit/login/

Then paste the API Token on that page below.

    Paste API Token from that page: cli-xxxx
>>> [4] <conduit> user.whoami() <bytes = 117>
>>> [5] <http> http://secure.phabricator.com/api/user.whoami
<<< [5] <http> 458,490 us
<<< [4] <conduit> 459,419 us
Usage Exception: The token "cli-xxxx" is not a valid API Token. The server returned this response when trying to use it as a token: ERR-INVALID-SESSION: Session key is not present.

[2016-07-22 16:44:35] EXCEPTION: (ArcanistUsageException) The token "cli-xxxx" is not a valid API Token. The server returned this response when trying to use it as a token: ERR-INVALID-SESSION: Session key is not present. at [<arcanist>/src/workflow/ArcanistInstallCertificateWorkflow.php:134]
arcanist(head=ccu, ref.master=6cb13cc16dc0, ref.ccu=8c016bb47f3f), phutil(head=ccu, ref.master=a6802e4d6b5f, ref.ccu=5fd2cf9d5ddd)
  #0 ArcanistInstallCertificateWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
zsh: exit 255   arc install-certificate http://secure.phabricator.com --trace
strictly:~% echo '{}' | arc --conduit-uri http://secure.phabricator.com/api/ --conduit-token cli-xxxx call-conduit user.whoami
Exception
ERR-INVALID-SESSION: Session key is not present.
(Run with `--trace` for a full exception trace.)
zsh: done       echo '{}' | 
zsh: exit 1     arc --conduit-uri http://secure.phabricator.com/api/ --conduit-token   
strictly:~% echo '{}' | arc --conduit-uri https://secure.phabricator.com/api/ --conduit-token cli-xxxx call-conduit user.whoami
{"error":null,"errorMessage":null,"response":{"phid":"PHID-USER-5atcce37qlwkkojcfupc","userName":"fooishbar","realName":"Daniel Stone","image":"https:\/\/p.phcdn.net\/file\/data\/@secure\/xy5sl74zq4z6fbskjevs\/PHID-FILE-pvhkk3dtagczflcpr5uq\/profile-10525781_10152557377385941_5502342738383595126_n.jpg","uri":"https:\/\/secure.phabricator.com\/p\/fooishbar\/","roles":["verified","approved","activated"],"primaryEmail":"daniel@fooishbar.org"}}

Why are you running arc install-certificate with a manually entered URL?

michaeljs1990 added a subscriber: michaeljs1990.EditedJul 22 2016, 5:15 PM

This looks closer to a server configuration issue to me and is actually a pretty strange behavior (mostly because i'm not sure how the redirect after it looks like a request is served works). If I go to http://secure.phabricator.com/api/user.whoami in the browser i get the following

{
result: null,
error_code: "ERR-CONDUIT-CORE",
error_info: "You are trying to save some data to Phabricator, but the request your browser made included an incorrect token. Reload the page and try again. You may need to clear your cookies. This was a Web request. This request had no CSRF token."
}

as expected or maybe not? However if i run curl http://secure.phabricator.com/api/user.whoami i get

{"result":null,"error_code":"ERR-INVALID-SESSION","error_info":"Session key is not present."}

fwiw this happens on our install as well however we haven't had any users run into it.

@epriestley Because it allows the reader of this report to reproduce the underlying issue without cloning anything or clobbering their .arcrc. Here are some less contrived scenarios, which are not mutually exclusive:

  • the user makes a typo (project lacking .arcconfig, or trying to use Phabricator for the first time themselves)
  • the .arcconfig comes from a time before Let's Encrypt (e.g. if you're in the middle of a git bisect that takes you back some distance)
  • the admin is just attempting to migrate people to more secure services and does not expect that services will fail (or at least not fail in such an opaque and confusing manner) in the face of a redirect

Specifically, I'd like to understand what concrete pathway led you to encounter this. A list of hypothetical situations isn't useful in anticipating or recovering from broader problems.

For example, if we receive several reports of similar issues which all have typos as the root cause, we might try both "http" and "https" and suggest to the user "Did you make a typo on the protocol?". If only a handful of users ever make this mistake, it might not be worth the complexity.

On the other hand, if we receive several reports of http -> https upgrades, we might look at different changes to try to anticipate or recover from those problems.

If things are coming out of git bisect, we might fall back to trying to read .arcconfig at HEAD of master if we fail with the working copy settings and notice it's an ancestor of master (or whatever). (Or just warn about this as a possible source of errors.)

My expectation is that users should normally run arc install-certificate with an explicit URL very rarely, since it should usually be read from .arcconfig. If users are doing this during first-time setup, the documentation might need to be rewritten to be more clear (but it seems like Arcanist Quick Start doesn't tell you to do this, at least).

fooishbar added a comment.EditedJul 22 2016, 5:29 PM

Specifically, I'd like to understand what concrete pathway led you to encounter this. A list of hypothetical situations isn't useful in anticipating or recovering from broader problems.

Pretty much all of the above hypothetical situations.

I use git worktree (yes, I know that's not supported by arc at all), and have an install which began as plain HTTP (not HTTPS) only. The install was migrated to HTTPS, and a redirect put in place so as not to break old URLs. The .arcconfig for the project I was using was not committed to the repository, because we have not yet committed as a project to switch to Phabricator, and that particular worktree happened to have an old version which referenced HTTP rather than HTTPS.

I was guiding someone through attempting to use git-phab for the first time, and as that particular worktree was the one I had open in the adjacent tab, that was the source of the .arcconfig I pasted him to use.

On the other hand, if we receive several reports of http -> https upgrades, we might look at different changes to try to anticipate or recover from those problems.

I would suggest that http -> https redirects are quite common even with sites which have not transitioned through an upgrade, e.g. http://secure.phabricator.com.

My expectation is that users should normally run arc install-certificate with an explicit URL very rarely, since it should usually be read from .arcconfig. If users are doing this during first-time setup, the documentation might need to be rewritten to be more clear (but it seems like Arcanist Quick Start doesn't tell you to do this, at least).

Anyone creating a project, or trying to use Phabricator with a particular project for the first time, will be creating .arcconfig, which is an opportunity to get this wrong.

My install hit this today: we're trying to upgrade from http -> https, but don't want to break users with the old http:// address in their .arcconfig (either because they haven't pulled master recently, are working on a different branch, are bisecting to a commit prior to the .arcconfig being updated, etc).

It's easy enough to workaround though: adjust your web server to issue a 307 redirect instead of 301/302. No idea if this is good practice or not, but it does keep the POST request data when arc retries with the new URL instead of discarding it. This seems to be where the "Session key is not present" error is coming from.