We've received three reports recently of arc emitting CURLE_SSL_CONNECTION_ERROR unconditionally on Yosemite Beta 6 (two users were definitely on this system; @chengyin, I'm assuming you are too).
Description
Description
Revisions and Commits
Revisions and Commits
Related Objects
Related Objects
Event Timeline
Comment Actions
lusmac:proj lliu$ arc tasks --trace libphutil loaded from '/Users/lliu/bin/libphutil/src'. arcanist loaded from '/Users/lliu/bin/arcanist/src'. Config: Reading user configuration file "/Users/lliu/.arcrc"... Config: Did not find system configuration at "/etc/arcconfig". Working Copy: Reading .arcconfig from "/Users/lliu/Developer/git/proj/.arcconfig". Working Copy: Path "/Users/lliu/Developer/git/proj" is part of `git` working copy "/Users/lliu/Developer/git/proj". Working Copy: Project root is at "/Users/lliu/Developer/git/proj". Config: Did not find local configuration at "/Users/lliu/Developer/git/proj/.git/arc/config". >>> [0] <conduit> conduit.connect() <bytes = 463> >>> [1] <http> https://phabricator.dev.oanda.com/api/conduit.connect <<< [1] <http> 6,353 us <<< [0] <conduit> 6,662 us [2014-08-19 20:54:35] EXCEPTION: (HTTPFutureCURLResponseStatus) [cURL/35] (https://phabricator.dev.oanda.com/api/conduit.connect) <CURLE_SSL_CONNECT_ERROR> There was an error negotiating the SSL connection. This usually indicates that the remote host has a bad SSL certificate, or your local host has some sort of SSL misconfiguration which prevents it from accepting the CA. If you are using a self-signed certificate, see instructions in "libphutil/resources/ssl/README". at [<phutil>/src/future/http/HTTPSFuture.php:363] #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:39] #1 Future::resolve(NULL) called at [<phutil>/src/future/FutureProxy.php:36] #2 FutureProxy::resolve() called at [<phutil>/src/conduit/ConduitClient.php:24] #3 ConduitClient::callMethodSynchronous(string, array) called at [<arcanist>/src/workflow/ArcanistWorkflow.php:351] #4 ArcanistWorkflow::authenticateConduit() called at [<arcanist>/scripts/arcanist.php:302]
Comment Actions
Nothing related was mentioned in DP6's release note: https://developer.apple.com/library/prerelease/mac/releasenotes/General/rn-osx-10.10/index.html
Nor did I see other reports on the upstream (not that I'm familiar with curl).
I've been using arc in an VM.
Comment Actions
Even setting CURLOPT_SSL_VERIFYPEER to false doesn't make the SSL error go away. I was able to get arc temporarily working again by not setting CURLOPT_CAINFO.
The .pem file could be incompatible with Yosemite and I wasn't able to drag it into my keychain nor export to .p12. Also curl works fine without specifying --cacert:
$ curl -v https://google.com/ * Hostname was NOT found in DNS cache * Trying 74.125.239.98... * Connected to google.com (74.125.239.98) port 443 (#0) * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA * Server certificate: *.google.com * Server certificate: Google Internet Authority G2 * Server certificate: GeoTrust Global CA * Server certificate: Equifax Secure Certificate Authority > GET / HTTP/1.1 > User-Agent: curl/7.37.1 > Host: google.com > Accept: */* > < HTTP/1.1 301 Moved Permanently < Location: https://www.google.com/ < Content-Type: text/html; charset=UTF-8 < Date: Tue, 19 Aug 2014 21:06:25 GMT < Expires: Thu, 18 Sep 2014 21:06:25 GMT < Cache-Control: public, max-age=2592000 * Server gws is not blacklisted < Server: gws < Content-Length: 220 < X-XSS-Protection: 1; mode=block < X-Frame-Options: SAMEORIGIN < Alternate-Protocol: 443:quic < <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/">here</A>. </BODY></HTML> * Connection #0 to host google.com left intact $ curl -v --cacert /usr/local/lib/phabricator/libphutil/resources/ssl/default.pem https://google.com/ * Hostname was NOT found in DNS cache * Trying 74.125.239.101... * Connected to google.com (74.125.239.101) port 443 (#0) * SSL: certificate verification failed (result: 5) * Closing connection 0 curl: (51) SSL: certificate verification failed (result: 5)
Curl stuff:
$ curl --version curl 7.37.1 (x86_64-apple-darwin14.0) libcurl/7.37.1 SecureTransport zlib/1.2.5 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz $ php -r 'print_r(curl_version());' Array ( [version_number] => 468225 [age] => 3 [features] => 33469 [ssl_version_number] => 0 [version] => 7.37.1 [host] => x86_64-apple-darwin14.0 [ssl_version] => SecureTransport [libz_version] => 1.2.5 [protocols] => Array ( [0] => dict [1] => file [2] => ftp [3] => ftps [4] => gopher [5] => http [6] => https [7] => imap [8] => imaps [9] => ldap [10] => ldaps [11] => pop3 [12] => pop3s [13] => rtsp [14] => smtp [15] => smtps [16] => telnet [17] => tftp ) )
Comment Actions
... and it looks like --cacert is old news: http://curl.haxx.se/mail/archive-2013-10/0036.html. So I'm not sure how arc worked on Mavericks unless the certs registered with the keychain changed in 10.10 DP6.
Comment Actions
See also T4781, for Mavericks, although it seemed like that calmed down after ~one initial report.
Comment Actions
"Me too" on this issue. curl -v --cacert ... produces:
... * SSL: certificate verification failed (result: 5) * Closing connection 0 curl: (51) SSL: certificate verification failed (result: 5)
While it works on Mavericks.
Comment Actions
The best workaround I can come up with here is something like:
- Detect that we're on OSX Yosemite Beta 6 (how?) or newer (also how?)
- If we are, don't set CAINFO. Instead, set a "Yosemite" flag.
- If the request fails because of certificate validation, check for the "Yosemite" flag.
- If the "Yosemite" flag is set, check if the request tried to customize certificate information.
- If they did, throw a tailored exception pointing them at http://curl.haxx.se/mail/archive-2013-10/0036.html, or some flavor thereof: "you tried to set XYZ as a cert, but that doens't work on OSX, you need to do this instead".
This will let administrators continue using existing certificate configuration which can apply to all non-Yosemite systems, and Yosemite users can get a more tailored message. Given the discussion on the cURL list, it seems unlikely that a less roundabout workaround is forthcoming.
Comment Actions
This might help with version detection, down to the beta release with that build timestamp.
$ php -r 'echo php_uname('r')."\n".php_uname('s')."\n".php_uname('v')."\n";' 14.0.0 Darwin Darwin Kernel Version 14.0.0: Sat Aug 9 00:14:02 PDT 2014; root:xnu-2782.1.80~2/RELEASE_X86_64
Comment Actions
Okay, I"ll throw up some kind of garbage hack based on that and someone on YB6 can vet it?
Comment Actions
I'll try to arc patch.
Also it could also be simpler to check for just Darwin 14.0 since it sounds like older versions of OS X (incl. 10.10 beta 1-5) may have ignored CAINFO anyway (and no one's going to care about the betas in a month).
Comment Actions
Do you get the correct error for:
libphutil/ $ ./scripts/test/http.php https://www.pcwebshop.co.uk/(Use that domain literally; it has a self-signed cert.)
Comment Actions
Array
(
[0] => HTTPFutureCertificateResponseStatus Object
(
[statusCode:HTTPFutureResponseStatus:private] => 1
[uri:HTTPFutureResponseStatus:private] => https://www.pcwebshop.co.uk/
[message:protected] => [Certificate/1] (https://www.pcwebshop.co.uk/) There was an error verifying the SSL Certificate Authority while negotating the SSL conncetion. This usually indicates you are using a self-signed certificate.
As of OSX Yosemite, certificates must be added to the OSX keychain. You can do this with `security add-trusted-cert` from the command line, or by visiting the site in Safari and choosing to trust the certificate permanently.
For more information, see instructions in "libphutil/resources/ssl/README".
[string:Exception:private] =>
[code:protected] => 0
[file:protected] => /usr/local/arcanist/libphutil/src/future/http/HTTPSFuture.php
[line:protected] => 373
[trace:Exception:private] => Array
(
[0] => Array
(
[file] => /usr/local/arcanist/libphutil/src/future/Future.php
[line] => 39
[function] => isReady
[class] => HTTPSFuture
[type] => ->
[args] => Array
(
)
)
[1] => Array
(
[file] => /usr/local/arcanist/libphutil/scripts/test/http.php
[line] => 45
[function] => resolve
[class] => Future
[type] => ->
[args] => Array
(
)
)
)
[previous:Exception:private] =>
)
[1] =>
[2] => Array
(
)
)Comment Actions
I'm not sure if this is related to our server setting or not
$ arc list Exception ERR-CONDUIT-CORE: Attempting to construct a query containing characters outside of the Unicode Basic Multilingual Plane. MySQL will silently truncate this data if it is inserted into a `utf8` column. Use the `%B` conversion to escape binary strings data. (Run with --trace for a full exception trace.)
I get the correct result in a VM for the same server/repo:
Linux packer-virtualbox-iso 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Comment Actions
The error I got turned out to be caused by my idiotic (or fabulous) attempt to use emoji as my hostname. The factory one more specifically - guess you guys at Phacility should jump on the boat too.
Comment Actions
i had dug into this myself, and discovered it was a bug with curl itself, and only for subdomains with wildcard certs. yosemite included php with the latest version of curl, so it inconveniently picked up this bug as well. see the bug on the curl sourceforge project: https://sourceforge.net/p/curl/bugs/1404/ and the fix on the curl github project: https://github.com/bagder/curl/pull/115
@epriestly, you should probably revert D10315 when there's an update to yosemite that pulls in curl v7.38.
(btw, hi @ide!)