Page MenuHomePhabricator
Create Task
Maniphest T6667

Destroying a repository does not remove the commit worker tasks from the queue
Closed, WontfixPublic
Actions

Description

This basically exhausts the queue because the commit worker tasks continuously fail forever and nothing else can ever be processed.

Related Objects

Event Timeline

hach-que created this task.Dec 1 2014, 5:07 AM
hach-que assigned this task to epriestley.
hach-que raised the priority of this task from to Needs Triage.
hach-que updated the task description. (Show Details)
hach-que added a subscriber: hach-que.
hach-que added a comment.Dec 1 2014, 5:11 AM

It looks like they don't actually fail forever, but with 20k commit tasks in the queue it still takes ages to go through each of them only to discover that they're no longer relevant.

epriestley closed this task as Wontfix.Dec 1 2014, 11:50 AM

If you're not at HEAD, there was a fix for large queues recently (see T6615).

It's expected that you'll get through ~100/sec of these per taskmaster (i.e., this is the actual rate of completion I realized with bin/worker flood + bin/phd debug taskmaster during D10895), and thus clear 20K in roughly 1 minute with 4 taskmasters. If you're seeing a dramatically slower rate than that at HEAD, we should fix whatever that underlying issue is, not try to dequeue things. See also T3554.

epriestley added a comment.Dec 1 2014, 10:21 PM

Similar tasks at HEAD (4 taskmasters on my laptop):

Screen_Shot_2014-12-01_at_2.19.49_PM.png (143×1 px, 22 KB)

hach-que added a comment.Dec 1 2014, 11:10 PM

We had the fix for large queues applied.

They've cleared themselves overnight, but I don't think we were seeing times as long as 2us (we certainly weren't getting through 370k in 15 minutes). It was probably closer to about 100 tasks every 2 minutes.

Phabricator · Built by Phacility