Page MenuHomePhabricator
Differential D10907

try to find duplicates using an analyzed elasticsearch field
AbandonedPublic
Actions

Authored by fabe on Nov 28 2014, 1:03 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Dec 20, 6:49 PM
Unknown Object (File)
Thu, Dec 19, 1:50 PM
Unknown Object (File)
Thu, Dec 19, 1:50 PM
Unknown Object (File)
Thu, Dec 19, 1:50 PM
Unknown Object (File)
Thu, Dec 19, 1:50 PM
Unknown Object (File)
Mon, Dec 16, 9:59 AM
Unknown Object (File)
Thu, Nov 28, 2:03 AM
Unknown Object (File)
Thu, Nov 28, 2:03 AM
View All Files
Subscribers
aklapper
epriestley
joshuaspence
WikiChad

Details

Reviewers
None
Group Reviewers
Blessed Reviewers
Maniphest Tasks
T4828: Suggest/propose possible duplicates when creating a new task
Summary

Ref T6656 use elasticsearch to find duplicates

Test Plan

use ./bin/search find_duplicates to try without any index changes.
Then do a ./bin/search find_duplicates --installmapping to change the mapping.
Then all tasks need to be reindexed (./bin/search index --type TASK) and then try
./bin/search find_duplicates --analyzed and compare with the initial result.
Language analyzed is hardcoded to english for now.

Diff Detail

Event Timeline

fabe updated this revision to Diff 26201.Nov 28 2014, 1:03 PM
fabe retitled this revision from to try to find duplicates using an analyzed elasticsearch field.
fabe updated this object.
fabe edited the test plan for this revision. (Show Details)
fabe added a task: T6656: Maniphest Reports break when you have 75k tasks.
Herald added a reviewer: Blessed Reviewers. · View Herald TranscriptNov 28 2014, 1:03 PM
Herald added a subscriber: epriestley. · View Herald Transcript
Harbormaster completed remote builds in B3147: Diff 26201.Nov 28 2014, 1:03 PM
fabe edited tasks, added T4828: Suggest/propose possible duplicates when creating a new task; removed T6656: Maniphest Reports break when you have 75k tasks.Nov 28 2014, 1:59 PM
WikiChad added a subscriber: WikiChad.Dec 18 2014, 8:39 PM

This should probably be rebased on top of the work in D10955: Properly create Elasticsearch index.

fabe added a comment.Dec 19 2014, 12:55 PM

This diff is not really meant to be merged. (And i'm not sure if can even create a diff on top of another not yet landed diff?)
But you're right that if we want the numbers to be correct you should apply the patch from D10955 and then this one on top
and then only run: ./bin/search find_duplicates
I'll just remove the mapping stuff from this diff.

fabe updated this revision to Diff 26454.Dec 19 2014, 1:03 PM
fabe edited edge metadata.
fabe added a task: T6656: Maniphest Reports break when you have 75k tasks.

remove mapping stuff

fabe removed a task: T6656: Maniphest Reports break when you have 75k tasks.Dec 19 2014, 4:14 PM
aklapper added a subscriber: aklapper.Dec 30 2014, 2:02 PM
joshuaspence mentioned this in T6853: "Open Revisions Affecting These Files" should weight severity of changes.Jan 2 2015, 10:14 PM
joshuaspence added a subscriber: joshuaspence.Feb 26 2015, 12:00 AM
fabe abandoned this revision.May 25 2015, 12:44 PM

Elasticsearch mapping is part of head now. So the rest is quite obsolete.

Revision Contents
Changeset List

Diff 26454

View Options

src/__phutil_library_map__.php

Loading...
View Options

src/applications/search/engine/PhabricatorSearchEngineElastic.php

Loading...
View Options

src/applications/search/management/PhabricatorSearchManagementFindDuplicatesWorkflow.php

Loading...
Phabricator · Built by Phacility