Page MenuHomePhabricator

Improve Conduit performance for custom fields
ClosedPublic

Authored by epriestley on Jul 31 2016, 3:46 PM.
Tags
None
Referenced Files
F14357246: D16351.diff
Fri, Dec 20, 3:25 AM
Unknown Object (File)
Thu, Dec 19, 10:03 AM
Unknown Object (File)
Mon, Dec 16, 6:01 PM
Unknown Object (File)
Fri, Dec 6, 11:32 PM
Unknown Object (File)
Wed, Dec 4, 3:23 PM
Unknown Object (File)
Tue, Dec 3, 2:10 PM
Unknown Object (File)
Fri, Nov 29, 1:52 PM
Unknown Object (File)
Thu, Nov 28, 11:27 AM
Subscribers
None

Details

Summary

Ref T11404. Depends on D16350.

Currently, custom fields can issue "N+1" queries in some cases, so querying 100 revisions issues 100 extra queries.

This affects all *.search endpoints for objects with custom fields, and some older endpoints (notably differential.query).

This change bulk loads "normal" custom fields, which gets rid of some of these queries. Instead of loading fields for each object, we build a big list of all fields and load them all at once.

The next change will tackle the remaining inefficient edge queries.

Test Plan
  • Configured a custom field with normal database storage in Differential.
  • Ran differential.query, looking at custom fields in results for correctness.
  • Ran differential.revision.search, looking at custom fields in results for correctness.
  • In both cases, observed queries drop from 3N to 2N (all the "normal" custom field stuff got bulk loaded).

Diff Detail

Repository
rP Phabricator
Branch
cfield2
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 13201
Build 16908: Run Core Tests
Build 16907: arc lint + arc unit

Event Timeline

epriestley retitled this revision from to Improve Conduit performance for custom fields.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added reviewers: chad, yelirekim.
  • Fix some array_mergev() junk I refactored into not working.
chad edited edge metadata.
This revision is now accepted and ready to land.Jul 31 2016, 4:23 PM
This revision was automatically updated to reflect the committed changes.