Let me know if you have any questions on this. You can find instructions on developing database patches (e.g., to add new tables) in this document:

http://www.phabricator.com/docs/phabricator/article/Database_Schema.html

This is a really exciting one...! :D

The general goal is to improve the performance of all blame operations, whether they're web-based or not. Having a blame cache will let us implement faster CLI operations ("arc blame"), but most of the benefit will probably come just from making the web view faster. For now, the integration point you should focus on is the web interface, since the other stuff doesn't exist yet.

The actual changes you'll make to interact with the blame cache should probably happen in DiffusionFileContentQuery and/or its subclasses. This class is responsible for issuing all the blame queries we currently execute. Since the cache should be VCS-agnostic, it should probably be implemented in the base class.

Basically, DiffusionFileContentQuery->loadFileContent() looks like this right now:

$content = $this->executeQuery();
// ... do encoding junk ...
return $content;

Instead, you want it to look like this (greatly simplified):

$content = $this->readCache();
if (!$content) {
  $content = $this->executeQuery();
  $this->writeCache($content);
}
// ... do encoding junk ...
return $content;

DiffusionFileContentQuery operates on a single path, so you shouldn't need to worry about mapping commits to paths. You can also ignore the "cache only when changed" part for now -- we'll have a less effective cache without it, but it will be fairly easy to add that later and it reduces the amount of work you need to do to get it working. It might be easier to get started by implementing the cache for only one VCS (e.g., implement in DiffusionGitFileContentQuery->executeQuery()) and then generalizing it once it works.

Does that make sense? Let me know if anything is unclear -- this code is structured a bit weirdly.

Had some free time today and implemented this. The improvement however is not as big as i expected.
Doing a blame on phabricators __phutil_library_map__.php will take ~20sec without and ~15sec with the cache on my dev vm.

Right now the cache is only updated when DiffusionFileContentQuery gets called with a needs_blame = true.
Still needs to be integrated into the commit parsing pipeline.

I would be interested in helping with benchmarks as desired. I have a Mercurial repository with commits dating back to 2004, ~97k commits in total with currently ~7.5k files. Though, I tried running hg blame on a few files which contain unmodified lines since as far back as 2009 and the blame happens near instantaneous.