Page MenuHomePhabricator

Reverse meaning of task priority column

Authored by epriestley on Nov 23 2014, 7:34 PM.
Referenced Files
Unknown Object (File)
Tue, Mar 21, 1:03 PM
Unknown Object (File)
Tue, Feb 28, 10:00 AM
Unknown Object (File)
Feb 13 2023, 11:49 PM
Unknown Object (File)
Feb 7 2023, 12:20 PM
Unknown Object (File)
Jan 31 2023, 10:28 PM
Unknown Object (File)
Jan 24 2023, 9:03 AM
Unknown Object (File)
Jan 19 2023, 7:29 AM
Unknown Object (File)
Jan 13 2023, 2:15 PM



Ref T6615. Mixing ASC and DESC ordering on a multipart key makes it dramatically less effective (or perhaps totally ineffective).

Reverse the meaning of the priority column so it goes in the same direction as the id column (both ascending, lower values execute sooner).

Test Plan
  • Queued 1.2M tasks with bin/worker flood.
  • Processed ~1 task/second with bin/phd debug taskmaster before patch.
  • Applied patch, took ~5 seconds for ~1.2M rows.
  • Processed ~100-200 tasks/second with bin/phd debug taskmaster after patch.
  • "Next in Queue" query on daemon page dropped from 1.5s to <1ms.

Diff Detail

rP Phabricator
Lint Not Applicable
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Reverse meaning of task priority column.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: btrahan.

mysql query performance is dark magic.

Yeah -- my mental model of indexes is that MySQL just builds some kind of fancy magical tree inside itself, so it followed that after it walked down the "leaseOwner = ?" part of the tree to get to the "leaseOwner = null" node it would be looking at a "priority = ?" subtree, and it could arbitrarily start at the left or right side to do the order by bit, and once it had walked down that subtree to the left-most or right-most node it would have the same opportunity to start at the left or right side of the "id = ?" sub-sub-tree at each priority node.

Clearly that's not what it actually does given that priority DESC, id ASC is >1500x slower than priority ASC, id ASC with 1.2M rows.


btrahan edited edge metadata.

My mental model is "you think you are in control, but who is in control of chaos? Better run EXPLAIN again."

This revision is now accepted and ready to land.Nov 24 2014, 6:31 PM
This revision was automatically updated to reflect the committed changes.