Page MenuHomePhabricator

MySQL fulltext index does not find 3-character words when ft_min_word_len=4
Closed, ResolvedPublic

Description

Actually the search engine doesn't return all elements about a search.

If I search for the word "bug" it displays "No search results" .

I have a lot of DIFFERENTIAL and AUDIT elements with "BUG" or "BUGGY" or anything else in title
I have a lot of branches / commits with BUG ...

And it doesn't find anything.

Is it normal?
Regards
Jeremy

Event Timeline

jeremy.cohen-solal raised the priority of this task from to Needs Triage.
jeremy.cohen-solal updated the task description. (Show Details)

You need to configure ft_min_word_len in MySQL. By default, it is 4, so 3-letter words are not indexed:

http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html#sysvar_ft_min_word_len

Make sure to REPAIR TABLE afterward, as per the note:

NOTE: FULLTEXT indexes must be rebuilt after changing this variable. Use REPAIR TABLE tbl_name QUICK.

Specifically:

REPAIR TABLE phabricator_search.search_documentfield QUICK;
epriestley renamed this task from Improve search engine to MySQL fulltext index does not find 3-character words when ft_min_word_len=4.Nov 19 2013, 11:46 AM
epriestley triaged this task as Wishlist priority.Nov 20 2013, 5:39 PM

We should add some kind of setup warning about this, but it's very low priority for new installs and not relevant if they don't use MyISAM fulltext indexes.

Showing the warning on the results page might make more sense; T2632 is similar.

Let's just do a setup warning, T2605 is very similar.

  • Add a configuration check for ft_min_word_len > 3 if the FULLTEXT index is active.
  • Suggest reducing the value to 3 and repairing the table.
  • Maybe put the table repairing in bin/search repair.

Wasn't this fixed? I can search for "css" on this very Phabricator install and it works! However, it still appears to not be working on our install of Phabricator (upgraded to the latest one yesterday again).

Ah sorry, I had missed comment #3 somehow.

BTW any reason that phabricator_search / search_documentfield is a MyISAM table instead of InnoDB like all the other ones? Amazon RDS doesn't like MyISAM tables (see "Q: What storage engines does Amazon RDS for MySQL support?" in http://aws.amazon.com/rds/faqs/).

InnoDB only added support for FULLTEXT indexes very recently (MySQL 5.6).

We are running on 5.6. Can I convert this table to InnoDB or will this create problems with the Phabricator's schemas and storage upgrade?

You can safely convert it. We will probably automatically convert it after T1191, which is largely going to take the form of defining abstract column and engine types and putting a layer of indirection underneath them that chooses the best available format.

Or, specifically: you can safely convert it if it's really a drop-in replacement for MyISAM. I haven't evaluated FULLTEXT support on InnoDB so I'm not sure how compatible the two are.

Thanks. Based on this, I will wait for T1191.