We're making changes to fulltext indexing which impact existing installs. Updating through these changes may take more time and attention than usual because search indexes need to be rebuilt. This task discusses these changes at a high level.
These changes are not included in stable for 2016 Week 48 (rPcfcc3b83). Upgrading past that (including to HEAD of master) will trigger the changes.
Summary for operations:
- Phabricator now manages table engines, and bin/storage upgrade will ALTER TABLE and change engines if any table is using an engine which Phabricator does not expect. Generally, this means that any manual engine changes you have made will now be undone.
- If InnoDB FULLTEXT indexes are available in MySQL (which is common), Phabricator will now change the fulltext tables to use InnoDB. See T11741 for broader discussion.
- This may take a long time if your install has a lot of data.
- After the engine change, search indexes also need to be rebuilt for stemming support (see T6740). Without this rebuild, search won't be able to find older results consistently.
If you operate a large install, you can minimize downtime associated with the migration like this:
- Warn users that search will be degraded for some time after the upgrade.
- TRUNCATE the phabricator_search.search_documentfield table to destroy all the data it contains. This will break search.
- Perform the upgrade. The engine change should execute quickly because no data needs to be migrated.
- Phabricator will now be usable again, except that global search won't return results.
- Run bin/search index --all --background --force to rebuild the entire search index. Search will gradually return to service as these tasks complete.
Benefits for operations:
- If InnoDB FULLTEXT is available, Phabricator now upgrades all tables to InnoDB.
- We saw various reports of these MyISAM tables crashing and requiring repair. Since (if possible) we no longer use MyISAM, this should no longer occur.
- ft_boolean_syntax no longer needs to be configured on any system. This is particularly relevant to RDS, where it could not be configured (see T10642).
- ft_min_word_len no longer need to be configured (if InnoDB FULLTEXT is available).
- ft_stopword_file no longer needs to be configured (if InnoDB FULLTEXT is available).
Benefits for users:
- Searches which return very large result sets should now execute faster.
- Results where the object title matches the query are now ranked higher.
- Search now uses stemming, and a query like "delete" can now find documents with the words "deletion", "deleting", "deletes", etc.
- Search should now find more relevant results when backed by Amazon RDS.
We'll continue tuning this, but future changes should be significantly less disruptive.