Page MenuHomePhabricator

Make Facts more modern, DRY, and dimensional
ClosedPublic

Authored by epriestley on Feb 18 2018, 3:31 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Apr 2, 1:12 PM
Unknown Object (File)
Sat, Mar 30, 6:56 PM
Unknown Object (File)
Sat, Mar 30, 6:56 PM
Unknown Object (File)
Sat, Mar 30, 6:56 PM
Unknown Object (File)
Sat, Mar 23, 5:43 PM
Unknown Object (File)
Sat, Mar 23, 5:42 PM
Unknown Object (File)
Wed, Mar 20, 6:29 PM
Unknown Object (File)
Wed, Mar 20, 6:29 PM
Subscribers

Details

Summary

Ref T13083. Facts has a fair amount of weird hardcoding and duplication of responsibilities. Reduce this somewhat: no more hard-coded fact aggregates, no more database-driven list of available facts, etc. Generally, derive all objective truth from FactEngines. This is more similar to how most other modern applications work.

For clarity, hopefully: rename "FactSpec" to "Fact". Rename "RawFact" to "Datapoint".

Split the fairly optimistic "RawFact" table into an "IntDatapoint" table with less stuff in it, then dimension tables for the object PHIDs and key names. This is primarily aimed at reducing the row size of each datapoint. At the time I originally wrote this code we hadn't experimented much with storing similar data in multiple tables, but this is now more common and has worked well elsewhere (CustomFields, Edges, Ferret) so I don't anticipate this causing issues. If we need more complex or multidimension/multivalue tables later we can accommodate them. The queries a single table supports (like "all facts of all kinds in some time window") don't make any sense as far as I can tell and could likely be UNION ALL'd anyway.

Remove all the aggregation stuff for now, it's not really clear to me what this should look like.

Test Plan

Ran bin/fact analyze and viewed web UI. Nothing exploded too violently.

Diff Detail

Repository
rP Phabricator
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

  • Fix up fact analyze calling an undefined method after refactoring.
This revision was not accepted when it landed; it landed in state Needs Review.Feb 19 2018, 8:05 PM
This revision was automatically updated to reflect the committed changes.