• Fact cursors are currently per-object. I think they should likely be per <object, engine> pair so that subsets can be rebuilt more quickly for testing/development and the failure of one engine will not stop all other engines from processing data. On the other hand, fact processing is currently pretty fast and I don't expect to write a ton of engines so I'm not sure how critical this really is. Engines likely don't care how cursors work unless they depend on facts extracted by other engines and need to guarantee they execute later.
• FactEngine->getFactSpecs() isn't really authoritative -- it takes a fact definition externally. I think this should become the authoritative generator. That is, this piece of code needs to go sooner rather than later:

$types = array(
  '+N:*',
  '+N:DREV',
  'updated',
);

• The factRaw table has a text32 column for fact type and two varchar(64) columns for PHIDs. These might be better handled by creating a separate dimension table to reduce the size of each row in the raw fact table. This probably isn't a big deal to do later, although it would be nice to do it in the shorter term. If this was moved to a dimension table, it's possible that a given PHID could have different dimensional representations when used by different facts. I think this is probably confusing and not useful, but something to keep in mind.
• The aggregation stuff is currently tough for me to wrap my head around and probably needs clearer use cases. I think "aggregation" also currently refers to a mixture of "aggregation across dimensions" (e.g., how much did scope increase today across all projects?) and "accumulation" (e.g., how many objects have ever been created since the beginning of time?). Since these are both just performance-oriented changes I expect to de-emphasize pursuing them until we can generate useful charts with performance issues, then move runtime aggregation/accumulation to the extractor.

One overarching issue here is that facts don't have a clear permissions model. It's bad if anyone can go look up the burndown chart for #critical-security-issues and then correlate that chart against tasks showing up in #weird-things-that-happen-when-you-hold-down-the-letter-a or whatever and get a hint about a security concern. In the general case, charts like "Salary by Employee", "Acuisitions Affecting our Stock Price", etc., are also potentially problematic.

It's also impractical to run policy checks on each datapoint, since the thing will be unusable if we do. This is inevitably going to mean that Facts is policy-violating to some degree.

Some approaches we could take include:

• To query datapoints by dimension (like a project or user), you must be able to see that dimension. But maybe this needs to be a separate/stronger check, since many users may be able to see #critical-security-issues, just not see the tasks it contains. And everyone can see other users.
• Actual facts (like "salary") could have policies. This is probably mostly relevant for third-party objects with particularly valuable data or third-party use of Facts as a charting application, both of which are probably far away.
• We can restrict the ability to generate charts with novel datasets, either in general ("by default, only administrators can create a new chart") or on a dimension or fact basis ("only users who can edit #critical-security-issues can create new charts using datasets which select dimensional data for it"). Users who could see the dimensions could still view charts created by someone else, they just couldn't go into Facts and whip up their own version of the chart.
• We could make charts that intentionally lie to users. However, applying deterministic noise which resists filtering seems difficult.
• We could restrict the ability to see granular data (e.g., only weekly aggregations or bigger) or to see recent data.

(A "create rough version" action which optionally: strips axis labels; mutates the data a bit randomly; anonymizes all dimensions; and re-renders the chart in a hand-drawn XKCD-style might be interesting -- like "I want to illustrate my point by showing an image ofthis in a blog post, but I don't want to criticize individuals or actually disclose the exact numbers".)

I'm just going to keep an eye on this for now and do the obvious things (e.g., you have to be able to see something to see dimensional facts about it) but this may need some kind of real solution before Facts can leave prototype.

See PHI385. I think I stabilized the dashboard arrangement issues in D19123, but the arrange workflows are still in somewhat rough shape.

Just to briefly touch this:

The actual fact extraction seems to mostly work correctly: https://secure.phabricator.com/fact/chart/?y1=tasks.count.create

There's also a page which shows which facts are extracted from each object to make debugging the extractors easier: https://secure.phabricator.com/fact/object/T13083/

I'm planning to do some work on the UI next so that it's possible to produce at least basic, plausibly-useful charts. The actual bit where you specify a chart (series, range/domain, colors, labels, title, etc.) might be a bit "big blob of JSON" for now. Ideally, I get this just good enough to replace Maniphest Reports, delete them, kill scope on Facts there, then build "Portals".

Not sure if it's expected in the current state, but here's one that doesn't seem well-formed: https://secure.phabricator.com/fact/chart/?y1=tasks.open-count.status

The task.open-count.status counts open tasks that are the result of a status change (an existing task being closed, or a closed task being reopened). Since it's much more common for existing open tasks to be closed than for existing closed tasks to be reopened, the series should go negative pretty quickly and stay there over time, more or less ending up with a big negative number of "number of tasks ever closed".

I think the UI just has problems drawing this right now. You can see that the Y axis has kind of tried to do something, but it's going in the wrong direction (with -6 at the top of the screen and a line going off into the sky). This might be a real issue with the data, too, but the UI is definitely getting something wrong, so I think it might not be any more complicated than fixing the UI to handle negative values properly.