Page MenuHomePhabricator

Implement a composite query which unifies cursor paging across subqueries
Closed, WontfixPublic

Description

For T5750, we need a CompositeQuery which can walk a single cursor across a collection of subqueries. The goal is to end up with an object which works something like this and produces the expected results:

$results = id(new CompositeQuery())
  ->addQuery(new PeopleQuery())
  ->addQuery(new ProjectQuery())
  ->executeWithCursorPager($pager);

This is complicated to build and requires at least some infrastructure cleanup. Concretely, it's currently not possible to implement getPagingValue() for a query like this, because the result (which is really something like "cursor material", not a "paging value") depends on which direction the cursor is iterating.

On the other hand, implementing this without being able to implement getPagingValue() seems like a dead end that will saddle us with weird limitations. In particular, I think it's a requirement to allow a CompositeQuery to be embedded inside another CompositeQuery rather than arbitrarily restricting them to be 1-ply deep.

More broadly, most paging/order/cursor code in Query subclasses is not robust, and many of the methods related to paging are confusing, too general, not general enough, generally don't make sense, etc. It's unclear how much work this needs.

Related Objects

StatusAssignedTask
Resolvedchad
ResolvedNone
Resolvedepriestley
DuplicateNone
ResolvedNone
Wontfixepriestley
Resolvedepriestley
Openepriestley
Resolvedepriestley
ResolvedNone
ResolvedNone
DuplicateNone
Wontfixepriestley
DuplicateNone
Resolvedepriestley
OpenNone
Wontfixepriestley
Resolvedepriestley
Resolvedchad
Resolvedepriestley
Wontfixepriestley
OpenNone
OpenNone
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Wontfixepriestley

Event Timeline

epriestley claimed this task.
epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a subscriber: epriestley.

I think I'm going to give up and use offset paging.

We can implement the original class (CompositeQuery) today with relatively few issues (although not "no issues"), but that isn't sufficient in the general case, because I also need some kind of ProxyQuery which turns each user into @user + projects(@user), and #project into #project + not(#project) + any(#project) + member(#project), and can page across them with a cursor.

That sounds pretty reasonable, but I haven't had success generating an implementation I'm comfortable with: you end up with cursors starting in the middle of pages of virtual results and a whole lot of other really horrible edge cases. The amount of time I'm spending on this is just not justified by the value of the feature. Realistically, no one is going to page through 20,000 users or projects by clicking "next" 200 times.