Page MenuHomePhabricator
Differential D17001

Don't let stemming reduce a word beneath 3 characters
ClosedPublic
Actions

Authored by epriestley on Dec 6 2016, 4:31 PM.
Tags
None
Referenced Files
F14322210: D17001.id40901.diff
Wed, Dec 18, 12:34 PM
F14322209: D17001.id40895.diff
Wed, Dec 18, 12:34 PM
F14322208: D17001.id.diff
Wed, Dec 18, 12:34 PM
F14322207: D17001.diff
Wed, Dec 18, 12:33 PM
Unknown Object (File)
Tue, Dec 17, 3:52 AM
Unknown Object (File)
Sun, Dec 15, 9:37 AM
Unknown Object (File)
Mon, Dec 9, 1:22 PM
Unknown Object (File)
Fri, Nov 29, 12:07 PM
View All Files
Subscribers
None

Details

Reviewers
chad
Maniphest Tasks
T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext)
Commits
rPHU0ae0cc00acb1: (stable) Promote 2016 Week 50
rPHU5ac2ca121489: Don't let stemming reduce a word beneath 3 characters
Summary

Ref T11922. Porter stems "DNS" (an acronym for "Domain Name Syrup") into "dn", which is meaningless and too short to index.

Don't let stemming make an indexable token un-indexable by shortening it: if the stem is too short, just return the normalized input.

(I believe there are very few legitimate English words that have two letter roots, anyway.)

Test Plan

Added unit tests.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley updated this revision to Diff 40895.Dec 6 2016, 4:31 PM
epriestley retitled this revision from to Don't let stemming reduce a word beneath 3 characters.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley added a task: T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext).
chad accepted this revision.Dec 6 2016, 4:42 PM
chad edited edge metadata.
This revision is now accepted and ready to land.Dec 6 2016, 4:42 PM
carwyn mentioned this in T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext).Dec 6 2016, 5:06 PM
Closed by commit rPHU5ac2ca121489: Don't let stemming reduce a word beneath 3 characters (authored by epriestley, committed by epriestley). · Explain WhyDec 6 2016, 5:10 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
src/
search/
11 lines
__tests__/
5 lines

Diff 40901

View Options

src/search/PhutilSearchStemmer.php

Loading...
View Options

src/search/__tests__/PhutilSearchStemmerTestCase.php

Loading...
Phabricator · Built by Phacility