Page MenuHomePhabricator

Remove "Arcanist Projects"
Closed, ResolvedPublic

Description

Long ago, Phabricator organized code primarily in terms of "Arcanist Projects". These were roughly human-readable labels for subdirectories in a repository, like "frontend-server" or "internal-tools". This was somewhat natural in large SVN repositories, where one repository often contained multiple logical projects (each with its own independent codebase and rulesets), and these were the first kind of repositories Phabricator worked with (at Facebook, circa 2007).

Subversion is much less popular now than it was in 2007, and Git and Mercurial are much more popular. With Git and Mercurial, it is far more common to organize code in many different repositories, where each repository is one logical project. In this case, the concept of "arcanist projects" isn't useful, because they always map directly to repository roots but add an unnecessary layer of indirection, complexity, and confusion. To make this problem worse, "projects" have become a much more core part of the product, and many users experience significant confusion when they learn that "arcanist projects" and "projects" are entirely separate things, even though they basically have the same name.

Phabricator's integration with repositories and tooling for segmenting rulesets for a codebase has improved over time, and we've reduced the importance of "arcanist projects". Circa 2011 they were mandatory; circa 2012-2013 they became optional; we are now removing them entirely. We believe they are virtually unused by modern installs and that installs with large SVN repositories now have other tools available which can serve the same purposes in almost all cases without requiring us to retain this confusing relic of a bygone era.


FAQ

What are "Arcanist Projects"? Are these the same as "Projects"? "Arcanist Projects" are not the same as "Projects". One of the reasons we are removing arcanist projects is that they are often a source of confusion, precisely because they have such a similar name. Arcanist projects were completely separate, and configured in the very old RepositoriesArcanist Projects UI. If you use Git or Mercurial and/or started using Phabricator in the last two years, there's a good chance you've never encountered them and this change may not affect you at all.

Are you removing "Projects"? No. Projects are a cornerstone of the product, and all our plans around them involve improving them and making them more useful.


Updating

How do I update .arcconfig? If you're using the old project.name or very old project_id keys in your .arcconfig, remove them. In most cases, arc will be able to figure out which repository it should use automatically. You can use arc which to verify this. If arc which can not identify the repository, use repository.callsign in .arcconfig to specify it explicitly. Refer to the output of arc which for more details.

How do I update Herald rules? Instead of using "Arcanist Project" rules, use "Repository" rules. You can add "Affected Paths" or "Affected Package" conditions to increase the power and specificity of "Repository" conditions. The Owners tool is badly overdue for modernization; if you run into difficulty using it to define packages in order to replace "Arcanist Project" rules, let us know.

How do I update symbol indexing? Symbols are now indexed per-repository. The indexing script uses repositories and configuration has moved to Diffusion. You may need to tweak any helper scripts you maintain. See T8235 for pending documentation updates.

Revisions and Commits

rPHU libphutil
D13002
rARC Arcanist
D12945
D13003
D12999
D12971
Audited
D12962
D12972
D12901
D12900
rP Phabricator
Abandoned
D12899
D13011
D13012
D13014
D12897
D13005
D13004
D12898
D12970
D13001
D12992
D12896
D12895
D12894
D12893

Related Objects

Event Timeline

joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added a project: Repositories.
joshuaspence added a project: Diffusion.
joshuaspence added a subscriber: joshuaspence.

Things that still use Arcanist Projects:

  • Releeph (I'm not even sure how to test this stuff)
  • Differential diffs (I'm not sure how we go about removing arcanist projects from diffs)
  • Diffusion lint messages (this should be converted to use repositories)

@epriestley, do you have any idea how Differential would work without Arcanist Projects? My only thought is that we could do some sort of Conduit call to determine the repository. Probably this should happen server-side I guess.

We have logic to determine the repository already (you can see it at the user-facing level in arc which, for example). You should just be able to remove projects completely, I believe.

epriestley renamed this task from Remove deprecated "Arcanist Projects" to Remove "Arcanist Projects".May 18 2015, 1:59 PM
epriestley updated the task description. (Show Details)
epriestley removed a project: Repositories.

will we need to break compatibility with users of existing arc versions?

No, we shouldn't. All of the changes that I have been making should be backwards compatible. Older arcanist clients will still submit arcanistProject as part of a diff, but it will be silently ignored server-side. We will need to wait a while before landing D12899 because this diff removes the arcanist.projectinfo Conduit endpoint (which is still in use until after D12962 and D12971).

A couple of issues that I have noticed though:

  • Attempting to arc patch a diff from repository X onto repository Y no longer produces a nice error message (some explanation in D12962). Previously, we would use the arcanist project name as a unique identify for the repository, so if the arcanist project name from the diff doesn't match the local arcanist project name, then the user is likely doing something wrong. It is not really possible for us to do this with repository PHIDs/callsigns because they are not portable across installs.
  • Arcanist has a magical reenter_if_this_is_arcanist_or_libphutil function which relies on the arcanist project name. We probably need a way to handle this as well.

For SVN, how will this differ from current workflow when we have two working diffs from same repo but different "projects"? Will we now need to specify --update and --revision for arc diff and arc commit where we wouldn't need to previously? That would be sad.

You shouldn't, no. arc uses other information (mostly working copy path on disk, IIRC) in that case.