Page MenuHomePhabricator

Cloning repositories over HTTPS may fail after compression changes
Closed, ResolvedPublic

Description

See https://discourse.phabricator-community.org/t/cannot-fetch-from-upstream-right-now/3785.

$ rm -rf tmp-spellbook && git clone http://epriestley:qwerqwer9@local.phacility.com/source/spellbook.git tmp-spellbook
Cloning into 'tmp-spellbook'...
fatal: unable to access 'http://epriestley:qwerqwer9@local.phacility.com/source/spellbook.git/': Error while processing content unencoding: invalid stored block lengths

This only affects HTTP (not SSH) and is likely related to gzip changes from T13507.

Event Timeline

epriestley triaged this task as Normal priority.Apr 25 2020, 2:29 PM
epriestley created this task.

The request sequence here is:

  1. git clone reaches a Phabricator web application node A.
  2. In a clustered repository, A proxies to repository node B.
  3. B responds to the request.
  4. A proxies the response back to the client.

In request (2), we now send Accept-Encoding: gzip, since A can decode gzip responses. This is new since T13507. When we receive the response from B in (3), we decode it and return it to the client (which may or may not be able to decode gzip responses). This might not be ideal (it may entail us using gzip on a poorly-compressible payload over a very high-bandwith internal link, or decompressing and recompressing a response if the client supports decoding gzip itself) but should all work conceptually.

The issue is that we don't replace the Content-Encoding or Content-Length headers, so the response in (4) may be uncompressed git-protocol data with the leftover pre-decoding headers from response (3). The simplest fix is to replace these headers (we already do this for Transfer-Encoding), and I think this fix is generally reasonable, so I'm going to pursue it for now.

In a perfect world, these primitives would probably have better concepts of which headers are invalidated by mutations to request content and better concepts of passing capabilities on behalf of a proxy client, but I think things are in good-enough shape that this stuff doesn't need to happen today.

I believe D21167 fixed this.