Page MenuHomePhabricator

Implement ngram search for File objects

Authored by amckinley on Apr 17 2017, 7:14 PM.
Referenced Files
Unknown Object (File)
Thu, Nov 30, 2:45 AM
Unknown Object (File)
Thu, Nov 16, 8:15 AM
Unknown Object (File)
Thu, Nov 16, 5:06 AM
Unknown Object (File)
Fri, Nov 10, 7:38 PM
Unknown Object (File)
Oct 19 2023, 4:08 AM
Unknown Object (File)
Oct 12 2023, 1:16 PM
Unknown Object (File)
Sep 7 2023, 12:27 AM
Unknown Object (File)
Sep 5 2023, 1:42 PM



Follows the outline in D15656 for implementing ngram search for names of File objects. Also created FileFullTextEngine, because without implementing PhabricatorFulltextInterface, ./bin/search complains that File is not an indexable type.

Test Plan
  • ran ./bin/storage upgrade to apply the schema change
  • confirmed the presence of a new file_filename_ngrams table
  • added a couple file objects
  • ran bin/search index --type file --force
  • confirmed the presence of rows in file_filename_ngrams
  • did a few keyword searches and saw expected results

Diff Detail

rP Phabricator
Lint Not Applicable
Tests Not Applicable

Event Timeline

5 ↗(On Diff #42576)

Need to remove this unless we want to force a reindex of all documents during storage upgrade.

3 ↗(On Diff #42576)

Not sure if this class needs to exist, but I couldn't force reindexing without implementing PhabricatorFulltextInterface.

Note that this also doesn't clean up ngrams once files are deleted. I'm going to follow along with D14846: Implement basic ngram search for Owners Package names to fix that.

This is right, except that we probably should't implement FulltextInterface or fulltext-index these. bin/search index was written before the ngrams stuff was written, let me see if I can send you a quick patch to let it work without needing fulltext.

The cleanup should happen automatically as a side effect of DestructionEngine, but neither PhabricatorFileTemporaryGarbageCollector nor PhabricatorFileDeleteController use the destruction engine properly (bin/remove destroy will). Swapping those to use a destruction engine should fix the ngrams stuff, maybe as part of T10828.

To use the destruction engine, replace this:


...with this:

$engine = new PhabricatorDestructionEngine();

D17704 should fix the bin/search index stuff -- it ended up easier than I thought, assuming it works.

  • removing FileFulltextEngine
  • removing migration to force file reindex
  • update File deletion logic to use DestructionEngine
This revision is now accepted and ready to land.Apr 18 2017, 12:35 AM
This revision was automatically updated to reflect the committed changes.