Implement an "Attachments" behavior for Conduit Search APIs
2160c4561990
Actions

Description

Implement an "Attachments" behavior for Conduit Search APIs

Summary:
Ref T9964. We have various kinds of secondary data on objects (like subscribers, projects, paste content, Owners paths, file attachments, etc) which is somewhat slow, or somewhat large, or both.

Some approaches to handling this in the API include:

Always return all of it (very easy, but slow).
Require users to make separate API calls to get each piece of data (very simple, but inefficient and really cumbersome to use).
Implement a hierarchical query language like GraphQL (powerful, but very complex).
Kind of mix-and-match a half-power query language and some extra calls? (fairly simple, not too terrible?)

We currently mix-and-match internally, with ->needStuff(true). This is not a general-purpose, full-power graph query language like GraphQL, and it occasionally does limit us.

For example, there is no way to do this sort of thing:

$conpherence_thread_query = id(new ConpherenceThreadQuery())
  ->setViewer($viewer)
  // ...
  ->setNeedMessages(true)
  ->setWhenYouLoadTheMessagesTheyNeedProfilePictures(true);

However, we almost never actually need to do this and when we do want to do it we usually don't really want to do it, so I don't think this is a major limit to the practical power of the system for the kinds of things we really want to do with it.

Put another way, we have a lot of 1-level hierarchical queries (get pictures or repositories or projects or files or content for these objects) but few-to-no 2+ level queries (get files for these objects, then get all the projects for those files).

So even though 1-level hierarchies are not a beautiful, general-purpose, fully-abstract system, they've worked well so far in practice and I'm comfortable moving forward with them in the API.

If we do need N-level queries in the future, there is no technical reason we can't put GraphQL (or something similar) on top of this eventually, and this would represent a solid step toward that. However, I suspect we'll never need them.

Upshot: I'm pretty happy with "->needX()" for all practical purposes, so this is just adding a way to say "->needX()" to the API.

Specifically, you say:

{
  "attachments": {
    "subscribers": true,
  }
}

...and get back subscriber data. In the future (or for certain attachments), true might become a dictionary of extra parameters, if necessary, and could do so without breaking the API.

Test Plan: