Page MenuHomePhabricator
Create Task
Maniphest T8380

Queueing search indexers is very slow
Closed, ResolvedPublic
Actions

Description

I am currently rebuilding our search index as we are moving to ElasticSearch. One thing I noticed is that the reindexing is very slow... it seems that operations are queued only slightly faster than they are being processed. The command has been running for roughly 36 minutes and has processed 14.2% of all indexes, which puts it at a rate of around 12 objects per second.

> ./bin/search index --all --background
Indexing 12 object of type CONP.
Indexing 20791 object of type DREV.
Indexing 24566 object of type TASK.
Indexing 4 object of type CDTL.
Indexing 132 object of type PROJ.
Indexing 146602 object of type CMIT.
Indexing 290 object of type USER.
Indexing 1 object of type MOCK.
Indexing 17 object of type WIKI.
[=========-                                                        ]  14.2%

After this 36 minutes, there are less than 10 tasks sitting in "Next in Queue".

Related Objects

Event Timeline

joshuaspence created this task.Jun 2 2015, 12:38 AM
joshuaspence raised the priority of this task from to Needs Triage.
joshuaspence updated the task description. (Show Details)
joshuaspence added a project: Search.
joshuaspence added a subscriber: joshuaspence.

It might be worth noting that I am running the ./bin/search index command on the same machine that is running our daemons.

chad added a subscriber: chad.Jun 2 2015, 1:59 AM

Is this different you think than T8306

joshuaspence added a comment.Jun 2 2015, 2:54 AM

Sorta... My issue isn't that indexing ElasticSearch is slow, rather my issue is that queueing tasks into the daemons seems to be slow (this shouldn't be touching ElasticSearch, also I believe this issue existed when we were using MySQL indexes as well).

joshuaspence added a comment.Jun 2 2015, 8:57 PM

Here's a profile: https://secure.phabricator.com/xhprof/profile/PHID-FILE-6qploks2cyzbwune3vki/

Nothing particularly interesting... most of the time is spent in MySQL. This was from running ./bin/search index --all --background --type PROJ.

joshuaspence added a comment.Jun 3 2015, 12:00 AM

Looks like --background is six times faster though: https://secure.phabricator.com/xhprof/profile/PHID-FILE-zy56ub2hhqqmrsgelvdl/

• Soperf added a mock: M1435: Brokens?.Jun 18 2015, 3:55 AM
joshuaspence closed this task as Resolved.Jul 23 2015, 9:05 AM
joshuaspence claimed this task.

Pretty sure this was fixed by D13519.

Phabricator · Built by Phacility