The current ls-remote script as written does not actually work; I think it only returns branches in the remote that:

The actual extension is a 100-line Python script...
The real Pandora's box this opens...

The real Pandora's box this opens is that now that arc is exposed to whatever issues arise with the raw Python/Mercurial API, there's a much weaker argument against not writing more extensions.

I put the extension in phabricator/ earlier by accident, but that was just me goofing my local paths. The extension is purely client side: hg can do ls-remote just fine (it's just "do hg pull, but stop half way through"), there's just no UI command for it.

Would the use of an extension make additional setup necessary for observed repositories vs. hosted repositories, or are observed repositories out of scope for these set of changes?

I am considering reviving the command server since I don't remember it having any critical failures, but offhand:

I'm going to try writing a Mercurial extension instead.

I believe this is roughly the form we end up with, which is completely ridiculous and extremely difficult to test or have any confidence in:

As a tool developer, this kind of thing is infuriating:

This is also made more complicated because Mercurial branches and bookmarks are all in a single shared namespace to some degree. I'm not sure what happens if you try to fetch a bookmark that conflicts with a local branch name (presumably: it renames it as "divergent"?) or try to fetch a branch which conflicts with a local bookmark name (presumably: fails in a way that's hard to detect?).

When the user specifies arc land --onto marker, I'd like to identify whether marker is a bookmark or branch.

Just extracting a few blocks of old code for reference:

I may actually make the minimum version whichever version has this fix:

Today, arc land in Mercurial does not support the "merge" land strategy.

I am dropping support for versions of Mercurial older than 2.1.1 (released in May, 2012). This is the first version that: (a) has phases and (b) does not exit 1 when hg pull succeeds but fetches no changes.

After changes connected to T13546, the git workflow now looks like this:

I'd propose for simplicity that the focus switch from largefiles to lfs support for Mercurial repositories. lfs has been added to Phabricator for Git already T7789 and it seems that there'd be less work involved to piggyback on that.

Mercurial 4.6 has added hg help internals.bundle2 which will render a book to your console with details about the format/protocol. I did a high-speed skim and then tried out hg help internals.changegroup on a whim which produced an even longer book about the stream protocol (SSH only maybe?).

Actually turns out that the work around isn't reliable - I still get that error occasionally (I'm using SSH)

In T9450#163090, @WTreutterer wrote:

The same problem occurred on my site with the latest Phabricator and the latest Mercurial 3.7.2 installed on both sides.

Interestingly, it can be cured setting a special development debug flag for in the Mercurial configuration file:
in /etc/mercurial/hgrc or in vcs-user's ${HOME}/.hgrc adding the following snippet

[devel]
bundle2.debug = true

caused a hg push to a hosted Phabricator repository to work without a problem. But do not ask me, why.

For recent versions of mercurial to work over http I had to force httpheader=1024 in code which filters bundle2 capabilities.

Presuming this is resolved since we've seen at least some confirmation that it fixed issues and aren't aware of any remaining outstanding problems.

I'm not totally sure all variants of this are fixed, but I don't know how to reproduce any remaining issues.

I filed a summary of this in the Mercurial upstream to waste someone else's time so I feel better:

This is an explicit behavior in Mercurial and dates from 2007:

The rule Git uses appears to literally be "does the filename include a space":

I'm going to close this since it was mostly answered and the remaining questions (about custom extension development) are outside the scope of modern support. See T13039 for a followup about numeric fields in Herald.

Ah, thanks! This is probably effectively covered by T9948 anyway -- one of the major changes for the Git flavor of that (T9657) was "put things back the way they were when anything goes wrong, even if we discard merge/rebase work", and that seems like a better behavior. I'll make a note there just in case.

I tried a few scenarios for this and wasn't able to reproduce

1. Single commit in diff that creates conflict
2. Multiple commits in diff that all create conflicts
3. Single commit in diff where first commit does not create conflict but second does

After D18857, I'm not aware of any remaining, reproducible issues with Mercurial. If you're still encountering protocol issues after upgrading through D18857, let me know how to reproduce the problem you're seeing.

T13036 is a more focused rebrand of this issue, since this has a lot of philosophy and not a lot of actionable reproduction steps. The three line fix in D18857 may have resolved these issues.

Mercurial's protocol negotiation presumably considers the size of the change being transmitted in selecting the protocol format

I think D18857 fixes the pipe issues. Here's the problem:

Here's some evidence [that filtering bundle2] doesn't work:
we attempt to filter the protocol and tell the client that we don't support bundle2

Actually, this is less crazy than I thought.

This appears to date back to the introduction of the feature in D5738, where I suggested we use ancestors() without a legitimate reason (or maybe very old Mercurial had weird behavior).

We can't reproduce this, and can't fix issues we can't reproduce.

That's a good point! I wish it was designed like that since the beginning. I guess it won't happen with the current compatibility rules since it is likely to break automation.

In theory, you could require --config appear between hg and foo in hg foo .... This is already a valid position for --config (for example, hg --config x=y foo is valid), and already not a valid position for foo flags (for example, hg --branch default log is not valid).

https://phab.mercurial-scm.org/D1483 should make it possible to use -- to defend against non-flag user input. For inputs that are flags, use the form --flag=X and avoid --flag X.

We'll use the hardened mode once it's available, but I don't think we expect to take any further action here until then.

This is now in master so I've made the task public.

I don't want to leave RCE in Phabricator for 3 weeks, so I'm planning to land, deploy and disclose some version of the patch above today.

(I change visibility for this to @epriestley, @amckinley and @durin42 for now -- note that I intend to eventually make this issue public once the fix hits the commit log, so don't stockpile all your 0-days here.)

I missed your question in (1). A typical scenario for this would be when working on a feature which requires some refactoring work to be done. A revision is created with just the refactoring work and then a dependent revision is made which includes the actual feature work. During feature work additional refactoring might be made and need to update to that changeset and either add/amend changes, rebase the dependent revision back on top. Having the bookmarks auto created would help especially when using arc:bookmark when updating revisions back to phab.

In T9548#229910, @indygreg wrote:

https://www.mercurial-scm.org/repo/hg/file/default/contrib/phabricator.py is a Mercurial extension that allows you to send a series of changesets to Phabricator by calling Conduit APIs directly. The extension is currently tailored for the use cases of the Mercurial project itself, isn't distributed with Mercurial, and may require Mercurial 4.3. And it obviously bypasses Phabricator's built-in Mercurial server. But if you are willing to live with the caveats, you may find it a suitable workaround.

See also PHI45.

The behavior may have changed, but the change is from "we sometimes silently do the wrong thing" to "we explicitly refuse to do the wrong thing".

Ah, it sounded like a regression from the report. I haven't tried to bisect to determine if that was true.

That is, specifically, it expected that hg commands do not work in Phabricator if it can not determine the version of hg, so this is not a bug. Ignoring the setup warning might mean "we used to do Mercurial stuff but don't anymore, leaving us with some archived Mercurial repositories which we don't really need to look at, so it's okay that hg commands won't be able to run".

The version is strictly required because different versions of hg use different command syntax. If we can not determine which version of hg is installed, we can not run hg commands.

There doesn't seem to be anything actionable remaining on our end.

This cropped up in the HN thread -- works in my browsers (although Phabricator does not recognize it as a valid link):

Thanks for the writeup :)

The reason the upstream projects aren't using -- is that it isn't portable. For example, Putty's ssh doesn't support it.

The full set of mitigations is now available in stable, and I've promoted 2017 Week 32 (Mid August).

See also this enormously valuable contribution I made to the Git LFS upstream in connection with T7789 some time ago: