Page MenuHomePhabricator
Differential D17001

Don't let stemming reduce a word beneath 3 characters
ClosedPublic
Actions

Authored by epriestley on Dec 6 2016, 4:31 PM.
Tags
None
Referenced Files
F13084421: D17001.diff
Wed, Apr 24, 10:59 PM
Unknown Object (File)
Fri, Apr 19, 2:31 AM
Unknown Object (File)
Fri, Apr 19, 2:31 AM
Unknown Object (File)
Fri, Apr 19, 2:31 AM
Unknown Object (File)
Wed, Apr 17, 2:31 PM
Unknown Object (File)
Tue, Apr 16, 8:30 PM
Unknown Object (File)
Thu, Apr 11, 3:51 AM
Unknown Object (File)
Tue, Apr 2, 10:19 AM
View All Files
Subscribers
None

Details

Reviewers
chad
Maniphest Tasks
T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext)
Commits
rPHU0ae0cc00acb1: (stable) Promote 2016 Week 50
rPHU5ac2ca121489: Don't let stemming reduce a word beneath 3 characters
Summary

Ref T11922. Porter stems "DNS" (an acronym for "Domain Name Syrup") into "dn", which is meaningless and too short to index.

Don't let stemming make an indexable token un-indexable by shortening it: if the stem is too short, just return the normalized input.

(I believe there are very few legitimate English words that have two letter roots, anyway.)

Test Plan

Added unit tests.

Diff Detail

Repository
rPHU libphutil
Branch
stem1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 14781
Build 19329: Run Core Tests
Build 19328: arc lint + arc unit

Event Timeline

epriestley updated this revision to Diff 40895.Dec 6 2016, 4:31 PM
epriestley retitled this revision from to Don't let stemming reduce a word beneath 3 characters.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: chad.
epriestley added a task: T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext).
chad accepted this revision.Dec 6 2016, 4:42 PM
chad edited edge metadata.
This revision is now accepted and ready to land.Dec 6 2016, 4:42 PM
carwyn mentioned this in T11922: Upgrading: Changes to Fulltext Indexing (InnoDB Fulltext).Dec 6 2016, 5:06 PM
Closed by commit rPHU5ac2ca121489: Don't let stemming reduce a word beneath 3 characters (authored by epriestley, committed by epriestley). · Explain WhyDec 6 2016, 5:10 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
src/
search/
11 lines
__tests__/
5 lines

Diff 40895

View Options

src/search/PhutilSearchStemmer.php

Loading...
View Options

src/search/__tests__/PhutilSearchStemmerTestCase.php

Loading...
Phabricator · Built by Phacility