Page MenuHomePhabricator

Upgrading: Fulltext Search Services
Closed, ResolvedPublic

Description

Overview

The way that external fulltext search engines are configured has changed in 2017 Week 13.

  • If you use MySQL search, you are not impacted by the configuration changes and do not need to take any action. The vast majority of installs use MySQL search, and MySQL search is recommended in most cases.
  • If you use Elasticsearch, configuration has changed. Version compatibility may also have changed. See below for details.

For all installs, see Optional Project/Repository Indexes below for some optional steps.

Elasticsearch: Configuration Changes

The way Elasticsearch is configured has changed. These configuration options have been removed:

  • search.elastic.host - Now configured with cluster.search.
  • search.elastic.namespace - Now configured with the "path" key in cluster.search. See the next section for discussion.

If you currently use Elasticsearch with a setting like http://elastic.example.com:9200/ for search.elastic.host, your new cluster.search setting should be:

[
  {
    "type": "elasticsearch",
    "hosts": [
      {
        "protocol": "http"
        "host": "elastic.example.com",
        "port": 9200
      }
    ]
  }
]

For more details, see Cluster: Search.

Elasticsearch: Paths

If you previously used search.elastic.namespace, set the value as "path", as a top-level key in the service definition:

[
  {
    "type": "elasticsearch",
    "path": "/phabricator2",
    "hosts": [
      ...
    ]
  }
]

Use of this option is very rare.

Elasticsearch: Versions

Phabricator has historically had uneven support for different versions of Elasticsearch. The updated code has primarily been developed and tested against Elasticsearch 5 (the most recent version, at time of writing) and will work best with a modern Elasticsearch server.

If you run Elasticsearch 2, you can add "version" as a top-level key in the service definition:

[
  {
    "type": "elasticsearch",
    "version": 2
    "hosts": [
      ...
    ]
  }
]

This is also supported, but has not been tested as thoroughly.

If you run an older version, you may find things don't work; you are probably best off upgrading to Elasticsearch 5. See the next section for guidance.

Elasticsearch: Hard Reset

If you'd like to upgrade to Elasticsearch 5 from an older version of Elasticsearch, or generally wipe the slate clean and start from scratch, do this:

  • Set up Elasticsearch 5.
  • Configure cluster.search to point at it.
  • Run bin/search init.
  • Run bin/search index --all --background --force to populate the index.

This last command can be run while Phabricator is online. For general guidance on rebuilding search indexes, see T11932.

Optional Project/Repository Indexes

Adjacent to these configuration changes, some document types now have slightly richer fulltext indexes. You may optionally reindex projects and repositories to take advantage of these updates. If you do, users may have better luck finding relevant project and repository results using global search. This impacts both MySQL and Elasticsearch. If you don't run these commands, the indexes will tend to naturally update over time on their own.

phabricator/ $ ./bin/search index --type project --force
phabricator/ $ ./bin/search index --type repository --force

These commands can be run while Phabricator is running and should complete quickly (a few minutes at most) for most installs.

Event Timeline

epriestley updated the task description. (Show Details)Apr 2 2017, 7:13 PM
epriestley moved this task from Backlog to v2 on the Search board.Apr 12 2017, 2:35 PM
epriestley closed this task as Resolved.Apr 12 2017, 2:43 PM
epriestley claimed this task.

This has been live for nearly two weeks without anything apparently exploding so we don't have any further acton planned upstream (although see T12450 for some followups). If you do run into issues, feel free to file new tasks describing them.

Slightest of issues with the repository indexing mentioned above:

luca:~/phabricator$ ./bin/search index --type repository --force
Usage Exception: Type "repository" matches multiple indexable objects. Use a more specific string. Matching object types are: PhabricatorRepository, PhabricatorRepositoryCommit.
luca:~/phabricator$

As of effectively HEAD (I have some extremely minor css changes and a remarkup latex renderer that I can't imagine are related to this):

phabricator 7aa111862df235c0d9628fd332a715de48a00917 (Sat, Jun 3) (branched from 17f092307cdfe4891ee66075e2665abb3578a976 on origin) arcanist 129d51fa0936c9bae48fadf3a3f39e26d69d24da (Thu, May 18)
phutil a900d7b63e954e221efe140f0f33d3d701524aae (Apr 23 2017)

It seems not to be the most critical of issues.