Page MenuHomePhabricator

Index only the first 1,000 comments on any object
ClosedPublic

Authored by epriestley on Jun 22 2018, 4:10 PM.
Tags
None
Referenced Files
F14058322: D19503.id46634.diff
Sun, Nov 17, 10:58 AM
F14039464: D19503.id46640.diff
Mon, Nov 11, 5:42 AM
F14018182: D19503.id46640.diff
Tue, Nov 5, 5:43 AM
F14009013: D19503.id46640.diff
Wed, Oct 30, 8:35 AM
F14007768: D19503.diff
Tue, Oct 29, 11:27 AM
F14005817: D19503.diff
Sun, Oct 27, 8:07 PM
F13994474: D19503.diff
Wed, Oct 23, 6:25 AM
F13990477: D19503.diff
Tue, Oct 22, 3:54 AM
Subscribers
None

Details

Summary

Depends on D19502. Ref T13151. See PHI719. An install ended up with an object with 111,000+ comments on it because someone wrote a script to treat it like a logfile.

Although we seem to do mostly okay with this (locally, it only takes about 30s to index a similar object) we'll hit a wall somewhere (since we need to hold everything in memory), and it's hard to imagine a legitimate object with more than 1,000 comments. Just ignore comments past the first thousand.

(Conpherence threads may legitimately have more than 1,000 comments, but go through a different indexer.)

Test Plan
  • Piped some comments into maniphest.edit in a loop to create a task with 100K comments.
  • Ran bin/search index Txxx --force to reindex it, with --trace.
    • Before: task indexed in about 30s.
    • After: script loaded comments with LIMIT 1000 and indexed in a couple seconds.

Diff Detail

Repository
rP Phabricator
Branch
search2
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 20440
Build 27763: Run Core Tests
Build 27762: arc lint + arc unit