Page MenuHomePhabricator
Differential D18687

Add a garbage collector for common ngrams
ClosedPublic
Actions

Authored by epriestley on Oct 5 2017, 12:27 AM.
Tags
None
Referenced Files
F15505894: D18687.diff
Tue, Apr 15, 6:48 AM
F15503475: D18687.id44868.diff
Mon, Apr 14, 10:07 AM
F15502429: D18687.id44864.diff
Mon, Apr 14, 12:38 AM
F15492686: D18687.id44862.diff
Sat, Apr 12, 5:10 PM
F15481264: D18687.id.diff
Tue, Apr 8, 7:31 PM
F15478347: D18687.diff
Mon, Apr 7, 10:11 PM
F15478162: D18687.id44868.diff
Mon, Apr 7, 8:52 PM
F15472866: D18687.id44862.diff
Sat, Apr 5, 5:05 PM
View All Files
Subscribers
None

Details

Reviewers
amckinley
Maniphest Tasks
T13000: Sustained MySQL I/O overwhelmed db009 / huge Ferret engine ngrams table
Commits
rP66df5b149309: Add a garbage collector for common ngrams
Summary

Ref T13000. After an ngram is marked as "common", we can delete it from the storage table.

Currently, the only way to get ngrams marked as "common" is to manually run bin/search ngrams, so this has no impact on normal installs.

Test Plan

Ran bin/garbage collect, saw it start chewing through my local Maniphest ngrams table and removing common ngrams.

Diff Detail

Repository
rP Phabricator
Branch
garbage1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 18641
Build 25113: Run Core Tests
Build 25112: arc lint + arc unit

Event Timeline

epriestley created this revision.Oct 5 2017, 12:27 AM
Harbormaster completed remote builds in B18640: Diff 44862.Oct 5 2017, 12:28 AM
epriestley updated this revision to Diff 44864.Oct 5 2017, 12:41 AM
  • Also add a little warning text to bin/search ngrams.
Harbormaster completed remote builds in B18641: Diff 44864.Oct 5 2017, 12:42 AM
amckinley accepted this revision.Oct 5 2017, 6:01 PM
This revision is now accepted and ready to land.Oct 5 2017, 6:01 PM
Closed by commit rP66df5b149309: Add a garbage collector for common ngrams (authored by epriestley). · Explain WhyOct 5 2017, 6:41 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
src/
2 lines
applications/
search/
garbagecollector/
55 lines
management/
5 lines

Diff 44864

View Options

src/__phutil_library_map__.php

Loading...
View Options

src/applications/search/garbagecollector/PhabricatorSearchFerretNgramGarbageCollector.php

Loading...
View Options

src/applications/search/management/PhabricatorSearchManagementNgramsWorkflow.php

Loading...
Phabricator · Built by Phacility