Page MenuHomePhabricator

Make Ferret indexing more robust (UTF8, exception handling)
ClosedPublic

Authored by epriestley on Aug 28 2017, 10:41 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Mar 27, 12:11 AM
Unknown Object (File)
Feb 3 2024, 4:34 PM
Unknown Object (File)
Jan 30 2024, 8:29 PM
Unknown Object (File)
Jan 20 2024, 7:37 AM
Unknown Object (File)
Jan 20 2024, 7:07 AM
Unknown Object (File)
Jan 20 2024, 6:46 AM
Unknown Object (File)
Jan 6 2024, 2:02 AM
Unknown Object (File)
Dec 30 2023, 2:35 PM
Subscribers
None

Details

Summary

Ref T12819. Two minor improvements from live data:

  • Tokenize in a UTF8-aware way.
  • When one document fails to index, kill the transaction explicitly (rather than leaving it hanging) so we don't cause other failures later.
Test Plan

Created some UTF8 documents locally, indexed them, got clean results.

Diff Detail

Repository
rP Phabricator
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

This revision is now accepted and ready to land.Aug 28 2017, 10:49 PM
This revision was automatically updated to reflect the committed changes.