Differential D18487

Make Ferret indexing more robust (UTF8, exception handling)
ClosedPublic
Actions

Authored by epriestley on Aug 28 2017, 10:41 PM.

Details

Reviewers

chad

Maniphest Tasks

T12819: InnoDB FULLTEXT appears to fail catastrophically once it reaches a moderate size

Commits

rP4005a465f7d0: Make Ferret indexing more robust (UTF8, exception handling)

Summary

Ref T12819. Two minor improvements from live data:

Tokenize in a UTF8-aware way.
When one document fails to index, kill the transaction explicitly (rather than leaving it hanging) so we don't cause other failures later.

Test Plan

Created some UTF8 documents locally, indexed them, got clean results.

Diff Detail

Repository

rP Phabricator

Branch

ferret2

Lint

Lint Passed

Unit

Tests Passed

Build Status

Buildable 18196
Build 24467: Run Core Tests
Build 24466: arc lint + arc unit

Event Timeline

epriestley created this revision.Aug 28 2017, 10:41 PM

Harbormaster completed remote builds in B18196: Diff 44409.Aug 28 2017, 10:41 PM

Harbormaster completed remote builds in B18196: Diff 44409.

chad accepted this revision.Aug 28 2017, 10:49 PM

This revision is now accepted and ready to land.Aug 28 2017, 10:49 PM

Closed by commit rP4005a465f7d0: Make Ferret indexing more robust (UTF8, exception handling) (authored by epriestley). · Explain WhyAug 28 2017, 10:50 PM

This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Path

Size

src/

applications/

search/

engineextension/

PhabricatorFerretFulltextEngineExtension.php

7 lines

ngrams/

PhabricatorNgramEngine.php

6 lines

Diff 44409

View Options

src/applications/search/engineextension/PhabricatorFerretFulltextEngineExtension.php