Page MenuHomePhabricator

Remove "Arcanist Projects"
Closed, ResolvedPublic

Assigned To
Authored By
joshuaspence
Mar 17 2015, 1:15 PM
Referenced Files
None
Tokens
"Manufacturing Defect?" token, awarded by chad."Yellow Medal" token, awarded by avivey.

Description

Long ago, Phabricator organized code primarily in terms of "Arcanist Projects". These were roughly human-readable labels for subdirectories in a repository, like "frontend-server" or "internal-tools". This was somewhat natural in large SVN repositories, where one repository often contained multiple logical projects (each with its own independent codebase and rulesets), and these were the first kind of repositories Phabricator worked with (at Facebook, circa 2007).

Subversion is much less popular now than it was in 2007, and Git and Mercurial are much more popular. With Git and Mercurial, it is far more common to organize code in many different repositories, where each repository is one logical project. In this case, the concept of "arcanist projects" isn't useful, because they always map directly to repository roots but add an unnecessary layer of indirection, complexity, and confusion. To make this problem worse, "projects" have become a much more core part of the product, and many users experience significant confusion when they learn that "arcanist projects" and "projects" are entirely separate things, even though they basically have the same name.

Phabricator's integration with repositories and tooling for segmenting rulesets for a codebase has improved over time, and we've reduced the importance of "arcanist projects". Circa 2011 they were mandatory; circa 2012-2013 they became optional; we are now removing them entirely. We believe they are virtually unused by modern installs and that installs with large SVN repositories now have other tools available which can serve the same purposes in almost all cases without requiring us to retain this confusing relic of a bygone era.


FAQ

What are "Arcanist Projects"? Are these the same as "Projects"? "Arcanist Projects" are not the same as "Projects". One of the reasons we are removing arcanist projects is that they are often a source of confusion, precisely because they have such a similar name. Arcanist projects were completely separate, and configured in the very old Repositories → Arcanist Projects UI. If you use Git or Mercurial and/or started using Phabricator in the last two years, there's a good chance you've never encountered them and this change may not affect you at all.

Are you removing "Projects"? No. Projects are a cornerstone of the product, and all our plans around them involve improving them and making them more useful.


Updating

How do I update .arcconfig? If you're using the old project.name or very old project_id keys in your .arcconfig, remove them. In most cases, arc will be able to figure out which repository it should use automatically. You can use arc which to verify this. If arc which can not identify the repository, use repository.callsign in .arcconfig to specify it explicitly. Refer to the output of arc which for more details.

How do I update Herald rules? Instead of using "Arcanist Project" rules, use "Repository" rules. You can add "Affected Paths" or "Affected Package" conditions to increase the power and specificity of "Repository" conditions. The Owners tool is badly overdue for modernization; if you run into difficulty using it to define packages in order to replace "Arcanist Project" rules, let us know.

How do I update symbol indexing? Symbols are now indexed per-repository. The indexing script uses repositories and configuration has moved to Diffusion. You may need to tweak any helper scripts you maintain. See T8235 for pending documentation updates.

Revisions and Commits

Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision

Event Timeline

joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added a project: Repositories.
joshuaspence added a project: Diffusion.
joshuaspence added a subscriber: joshuaspence.
joshuaspence added a revision: Restricted Differential Revision.May 6 2015, 11:58 AM

Things that still use Arcanist Projects:

  • Releeph (I'm not even sure how to test this stuff)
  • Differential diffs (I'm not sure how we go about removing arcanist projects from diffs)
  • Diffusion lint messages (this should be converted to use repositories)

@epriestley, do you have any idea how Differential would work without Arcanist Projects? My only thought is that we could do some sort of Conduit call to determine the repository. Probably this should happen server-side I guess.

We have logic to determine the repository already (you can see it at the user-facing level in arc which, for example). You should just be able to remove projects completely, I believe.

joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 12:32 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:03 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:10 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:24 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:26 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:39 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 1:45 PM
epriestley renamed this task from Remove deprecated "Arcanist Projects" to Remove "Arcanist Projects".May 18 2015, 1:59 PM
epriestley updated the task description. (Show Details)
epriestley removed a project: Repositories.
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 2:09 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 18 2015, 2:17 PM
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a commit: Restricted Diffusion Commit.May 18 2015, 2:37 PM
joshuaspence added a revision: Restricted Differential Revision.May 18 2015, 2:39 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 19 2015, 9:28 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 20 2015, 1:31 AM
joshuaspence added a revision: Restricted Differential Revision.May 20 2015, 1:45 AM
joshuaspence added a revision: Restricted Differential Revision.May 21 2015, 9:11 AM
joshuaspence added a revision: Restricted Differential Revision.May 21 2015, 9:06 PM
joshuaspence added a revision: Restricted Differential Revision.May 21 2015, 9:17 PM
joshuaspence added a revision: Restricted Differential Revision.May 21 2015, 9:19 PM

will we need to break compatibility with users of existing arc versions?

No, we shouldn't. All of the changes that I have been making should be backwards compatible. Older arcanist clients will still submit arcanistProject as part of a diff, but it will be silently ignored server-side. We will need to wait a while before landing D12899 because this diff removes the arcanist.projectinfo Conduit endpoint (which is still in use until after D12962 and D12971).

A couple of issues that I have noticed though:

  • Attempting to arc patch a diff from repository X onto repository Y no longer produces a nice error message (some explanation in D12962). Previously, we would use the arcanist project name as a unique identify for the repository, so if the arcanist project name from the diff doesn't match the local arcanist project name, then the user is likely doing something wrong. It is not really possible for us to do this with repository PHIDs/callsigns because they are not portable across installs.
  • Arcanist has a magical reenter_if_this_is_arcanist_or_libphutil function which relies on the arcanist project name. We probably need a way to handle this as well.
joshuaspence added a commit: Restricted Diffusion Commit.May 22 2015, 7:09 AM

For SVN, how will this differ from current workflow when we have two working diffs from same repo but different "projects"? Will we now need to specify --update and --revision for arc diff and arc commit where we wouldn't need to previously? That would be sad.

You shouldn't, no. arc uses other information (mostly working copy path on disk, IIRC) in that case.

joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 8:31 AM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 8:42 AM
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 8:59 AM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 9:11 AM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 10:50 AM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 12:06 PM
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 12:18 PM
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 12:29 PM
joshuaspence added a revision: Restricted Differential Revision.
joshuaspence added a revision: Restricted Differential Revision.
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 12:35 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 12:42 PM
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 12:48 PM
joshuaspence added a revision: Restricted Differential Revision.
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 12:53 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 9:08 PM
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a commit: Restricted Diffusion Commit.May 25 2015, 9:10 PM
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a commit: Restricted Diffusion Commit.
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 9:16 PM
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 9:22 PM
joshuaspence added a revision: Restricted Differential Revision.May 25 2015, 10:13 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 26 2015, 1:16 PM
joshuaspence added a commit: Restricted Diffusion Commit.May 31 2015, 11:54 PM
joshuaspence added a commit: Restricted Diffusion Commit.Jun 1 2015, 5:35 AM
joshuaspence closed this task as Resolved by committing Restricted Diffusion Commit.Jul 8 2015, 9:59 AM
joshuaspence added a commit: Restricted Diffusion Commit.