Page MenuHomePhabricator

2020 Week 16 (Late April)
Updated 1,429 Days AgoPublic

Summary of changes from February 12, 2020 to April 24, 2020.

CodebaseRepositoryHEADActivity
PhabricatorrPrPd05d8f655890 commits
ArcanistrARCrARC68f050bd87 commits
Instances (SAAS)rSAASrSAASf0581871 commit
Services (SAAS)rSERVICESrSERVICES280ff192 commits
Core (SAAS)rCORErCORE1d9333a10 commits
  • These changes were promoted to stable.
This release includes substantial changes.

General

  • The minimum version of PHP required to run Phabricator is now PHP 5.5.
  • The libphutil/ library is no longer distributed:
    • The client-side parts have merged into arcanist/.
    • The server-side parts have merged into phabricator/.
  • Revisions are now submitted as "drafts". Once unit tests pass, they are sent for review.
  • Some arc flags and workflows have changed, and a few have been removed.
  • JIRA integration has undergone major compatibility changes since the last release.

See below for more detail.

Minimum Version: PHP 5.5

Phabricator now requires a minimum version of PHP 5.5 on both the client and server.

In most cases, this should not be an especially onerous requirement. PHP 5.5 is a very old version of PHP: it was released in 2013, reached its end-of-life in 2016, and has been unsupported by the PHP upstream for almost four years.

Requiring this minimum version of PHP primarily allows use of yield, which supports writing some primitives for managing parallel service calls which would otherwise be very challenging to write maintainably. See T11968 for details.

If you're upgrading anyway, it's reasonable to upgrade past PHP 5.5: Phabricator runs well under PHP 7 and all development has been done under PHP 7 for several years.

End of libphutil/

See T13395 for details. Until now, Phabricator was distributed in three different repositories:

  • phabricator/, with server-side code;
  • arcanist/, with client-side code; and
  • libphutil/, with shared code.

In the modern codebase, this separation comes with more costs than benefits. In particular, it makes installing and managing arcanist/ on clients more complicated than it has to be, since you also have to deal with a separte repository in libphutil/.

The parts of libphutil/ used by arcanist/ have been moved there, and the rest of it has been moved to phabricator/. All code in libphutil/ has been deleted.

After successfully upgrading, you can delete libphutil/ and forget it existed.

If you have extension or third-party code that includes libphutil/ directly (this is very rare), it can be updated to include arcanist/ instead. In general, all the old classes and features still exist and you can get the same behavior by using Arcanist as a library, the code has just moved.

Draft Mode

See T13010. Revisions are now submitted as "drafts", which do not send email. Once builds pass, the revision is "published" and notifies reviewers that it's ready for them to look at.

Previously, reviewers would sometimes get an email requesting a review, begin reviewing the code, and then builds would fail. This feels bad for reviewers (your work is partly "wasted") and authors (you "wasted" reviewers' time) and is generally negative for everyone.

Under the new system, the review waits for builds to pass before it notifies reviewers.

This "publishing" step requires that the revision have builds, so if you don't use Harbormaster to trigger builds, all of the new stuff is skipped and revisions are sent to reviewers immediately (just like they were before).

You can also configure this behavior for builds on a build-by-build basis:

  • If you have flaky builds and don't want failed builds to return changes to the author, you can configure the builds to hold changes in the Draft state "If Building". Revisions will wait for builds to finish, but still notify reviewers even if the build fails.
  • If you have very long-running builds, you can configure the them to "Never" hold revisions. Reviewers will be notified immediately, as they were prior to this change.

Alongside this change, some changes to arc flags have landed. In particular, the --plan-changes and --preview flags have been replaced with the new --draft flag (which holds revisions in a "Draft" state even once builds pass).

arc Changes

See T13488 for details. arc has undergone major changes, although most of them are under the covers. Some flags and workflows have been tweaked or removed, and arc is now stricter about argument parsing.

  • When arc is run noninteractively, it now requires a -- flag terminator argument. See T13491 for details.

JIRA Integration

See T13493 for technical details. JIRA's API for identifying accounts has changed, and Atlassian has recently deployed some of these changes to their hosted JIRA instances.

Briefly, JIRA previously identified accounts with a human-readable primary key like alice@company.com. It now identifies accounts with a globally unique accountId -- and, in newer versions, no longer emits the old key. This is a good change conceptually, but the specific implementation appears to make it impossible for Phabricator to reliably track accounts across this change in the general case.

Phabricator has been (mostly?) updated for these changes but there is no clear path forward across all possible JIRA/Phabricator upgrade pathways. That is, newer versions of Phabricator can not implement any behavior which converges to the right state: situations exist where Phabricator may see a user log in as alice@company.com one day and aaaa-bbbb-1234 the next day and have no way to ever learn the mapping between those two accounts.

When Phabricator can learn the mapping, it will now learn the mapping. In other cases, this workflow will work in the general case:

  • Upgrade Phabricator: old versions of Phabricator do not know how to read the new accountId identifier.
  • Log in to Phabricator, using email account recovery if necessary.
  • In SettingsExternal Accounts, unlink the JIRA account.
    • This will make Phabricator "forget" the old account key.
  • Relink the JIRA account.
    • This will make Phabricator "learn" the new accountId.

More aggressive migrations are possible if you can generate a mapping of key values to accountId values from JIRA, but they currently require manual intervention. If you're a Phacility customer, file an issue on your support channel if you'd like a hand with this.

Operational Guidance

This upgrade can be performed in smaller pieces:

  • You can upgrade PHP separately, before upgrading Phabricator. Versions of Phabricator from 2013 or newer should run under PHP 5.5. Versions of Phabricator from 2018 or newer should run under PHP 7. After upgrading PHP, you can then upgrade Phabricator.
  • You do not need to upgrade arc (on client machines) and Phabricator (on server machines) at the same time. Older versions of either will generally work well with newer versions of the other, although you may encounter a few minor compatibility issues as versions diverge (or if one or the other is several years old). (Note that when you upgrade phabricator/, you do still need to upgrade the arcanist/ directory next to it on disk!)
  • You do not need to upgrade all arc clients at the same time. If your users manage their own arc installs, they can upgrade at their leisure.

So a reasonable upgrade plan might look like this:

  • If necessary, plan and execute an upgrade to PHP 5.5 or PHP 7. This upgrade is usually easy, even if you have patches or custom code and are upgrading all the way to PHP 7. Behaviors that have changed in PHP 7 are mostly managed by Phabricator (like signal and exception handling) or rarely impact program behavior (like the stability of sort()).
  • Wait until you're satisfied that things are stable.
  • Plan and execute a server upgrade, bringing arcanist/ and phabricator/ to modern stable. These changes will be user visible (particularly "draft" mode and UI changes to Differential).
  • Wait until you're statisfied that things are stable.
  • Eventually, delete libphutil/ on the servers.
  • Execute or encourage client upgrades.

You can be more or less formal about these steps as appropriate for your organization's size and level of process around tooling changes.

Old Branches

The release prior to this one is tagged as legacy-2019 in all three repositories. You can use this branch if you need a convenient way to identify the last version of Phabricator that works on older PHP, has not merged libphutil/ into the other repositoreis, does not include the arc infrastructure changes, etc.

Security

  • Fix an issue where raw diffs that are not attached to revisions (this is usually uncommon) skipped repository policy checks.
  • The notification.servers configuration option is now "Hidden", and requests to the notification server no longer follow redirects. These changes defuse a very-low-severity SSRF attack which could only be performed by administrators. Administrators can generally still perform similar attacks in other ways and these attacks are very difficult to prevent entirely, since making requests to internal services is part of Phabricator's expected behavior for authentication, repositories, etc.

Upgrading / Compatibility

  • See the discussions above.
  • [] There is a new setup warning which detects webserver mangling of "Content-Encoding: gzip" requests, most likely caused by the "SetInputFilter DEFLATE" directive in Apache. Phabricator is not compatible with this directive and it's probably a bad idea to use it the general case (the filter emits requests where "Content-Encoding" and "Content-Length" no longer match the response body). See T13507 for discussion.

Differential

  • [] Revisions are now created a "drafts" by default. See above for more details.
  • [] Changeset view state ("Highlight As...", "Hidden", etc.) is now persistent.
  • [] Open in External Editor
    • The "External Editor" settings now have their own page in Settings.
    • The "Edit Multiple Files" editor setting was removed (it had no effect).
    • The "Editor Link" setting is now a bit more strict, and a bit clearer with errors.
    • Fixed a bug where files containing the literal sequence %l didn't work.
    • Added an "Open in External Editor" keyboard shortcut (\).
    • Editor links now support %n (short name) and two stable identifiers (%d for ID and %p for PHID).
  • [] File Tree / Paths Panel
    • The "File Tree" setting has been removed. It is now always enabled for everyone. By default, it is collapsed and accessible by clicking a tab to exapand it (or by pressing f).
    • The UI now calls this element the "Paths Panel", but the code still calls it the "File Tree". Unclear which way the wind is blowing.
    • This UI has been reworked and it should generally be more useful and less glitchy.
  • [] Added some more keyboard commands and keyboard command hints.
  • [] Fixed a bug where Jupyter files with a trailing newline weren't rendered correctly in the diff engine.
  • [] If a file in an interdiff was not changed, but has inline comments, but the viewer has collapsed all of the comments, it is now hidden by default. Previously, it was shown by default.

Minor

  • [] The protocol vscode:// is now whitelisted as an Editor protocol.
  • Fixed an issue where certain Chinese glyphs disrupted tokenizer layouts in some versions of Firefox.
  • In relatively recent versions of Git, raw commit messages are now extracted more faithfully from the repository.
  • Phabricator can now parse JIRA project keys which contain numbers.
  • [] Performance of the "mark unreachable" operation in the repository daemons has been improved. Previously, the daemons could take O(N * M) time to mark a chain of N commits unreachable if M branches pointing near the tip of the N chain were simultaneously deleted. They now take O(N) time.
  • Fixed a bug where bin/remove destroy M123 would fail to destroy mocks.
  • Fixed some issues with the Diviner UI and multiple @task attributes.
  • [] Inline comments now report an isDone property in transaction.search, corresponding to the (published) checkbox state.
  • More *.search API calls now return a uri field.
  • [] bin/conduit call now supports a --local flag to
  • Two very old migrations (circa 2014) have been updated to correct issues (caused by later changes) that could make them fail on particular datasets.
  • [] Putting your cursor at the very bottom of a menu icon in Chrome no longer makes it vibrate rapidly.
  • Owners packages are now marked "Archived" in tokenizers.
  • [] The Aphlict log is now much less verbose when not running in "debug" mode.

Ferret (Fulltext Search)

  • [] Ferret now supports a "field present" operator (field:~) and a "field absent" operator (field:-). For example, you can find tasks with no description by searching for body:- in Maniphest.
  • Some other search syntax has changed slightly. In most cases, it is more strict than it was previously, and raises more errors for invalid or ambiguous input. For example, title:="" ("title exactly matches the empty string") was previously ignored as a term and is now a query syntax error.
  • [] Paste now supports fulltext search with Ferret.
  • Fixed issues where multibyte unicode characters were not split correctly for older (pre-Ferret) ngram indexes. Some multi-character and normalization behavior is still incorrect: for example, the sequences <LATIN SMALL e> + <COMBINING ACUTE> and <LATIN SMALL e WITH ACUTE> are still seen differently by the engine but do not appear different to users. This is partly a limitation of utf8_unicode_ci collation in MySQL, and will be labor-intensive to work around. See T13501.
  • Fixed an issue where very short search sequences (1 or 2 characters) incorrectly used the ngram index and excluded too many matches under older (pre-Ferret) searches. For example, searching for om now finds random stuff.txt in Files, but previously did not.

Arcanist

  • Many arc behaviors have changed. See T13488.
  • arc diff now tries to push LFS artifacts to staging areas.
  • [] Conduit requests and responses now support compression.
  • The wilds-win branch has been deleted.
  • The arcpatch-D14363 branch has been deleted. This branch was an error.
  • The conduit-4, conduit-5, and conduit-6 tags have been deleted. Arcanist moved away from explicit protocol versioning to feature detection around 2012.
  • The experimental branch has largely merged into master and will be deleted in a future release.
  • The wilds branch has partially merged into master and will eventually suffer a similar fate.
  • Old stable is available on the legacy-2019 branch.

libphutil

  • Changes are no longer distributed in libphutil/, and master and stable are now empty.
  • The conduit-4 and conduit-5 tags have been deleted.
  • Old stable is available on the legacy-2019 branch.

Internals

  • The Future API has changed in several ways.
    • If you extend Future (this is very rare), you likely need to make adjustments for the new class API.
    • If you use Future in a complex way, you may make adjustments. In particular, if you previously used FutureIterator to manage a dynamic, persistent pool of futures, you should likely move to FuturePool.
    • If you use Future in the most common ways, you probably aren't affected.
  • [] All PHP language-level errors are now raised as exceptions by default.

The [] icon indicates a change backed by support mana.

Last Author
epriestley
Last Edited
Apr 30 2020, 3:49 AM

Event Timeline

Spotted a typo, arcainst :)

After successfully upgrading, you can delete libphutil/ and forget it existed.

I've more than once found newcomers to open-source projects that use Differential struggle with getting this one right. This will help a lot. Thank you!