HomePhabricator

Add a workflow for populating (or depopulating) the common ngrams table

Description

Add a workflow for populating (or depopulating) the common ngrams table

Summary:
Depends on D18672. Ref T13000. This does an on-demand build of the common ngrams table.

Plan here is:

  • Push to secure.
  • Build the common ngrams table here.
  • See if stuff breaks?

If it looks okay on this dataset, we can build out the GC support and try it in production.

Test Plan:

  • Locally, my dataset has a bunch of bin/lipsum tasks with similar, common words.
  • Verified that ipsum terms now skip ngrams. For "lorem ipsum" search performance actually IMPROVED by skipping the ngrams table (12s to 9s).
  • Queried for normal terms, got very fast results using the ngram table, as normal.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13000

Differential Revision: https://secure.phabricator.com/D18673