Page MenuHomePhabricator

Increase the size of the Diffusion commit cache
ClosedPublic

Authored by epriestley on Feb 24 2017, 6:05 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Apr 10, 3:01 AM
Unknown Object (File)
Wed, Apr 10, 12:40 AM
Unknown Object (File)
Tue, Apr 9, 11:44 PM
Unknown Object (File)
Tue, Apr 9, 11:05 PM
Unknown Object (File)
Tue, Apr 9, 8:52 PM
Unknown Object (File)
Mon, Apr 8, 10:40 AM
Unknown Object (File)
Fri, Apr 5, 7:19 AM
Unknown Object (File)
Fri, Apr 5, 1:21 AM
Subscribers
None

Details

Summary

Ref T12296. This cache is used to cache Git ref heads (branches, tags, etc). Reasonable repositories may have more than 2048 of these.

When we miss the cache, we need to single-get refs to check them, which is relatively expensive.

Increasing the size of the cache to 65535 should only require about 7.5MB of RAM.

Additionally, fill only as much of the cache as actually fits. The FIFO nature of the cache can get us into trouble otherwise.

If we insert "A, B, C, D" and then lookup A, B, C, D, but the cache has maximum size 3, we get this:

  • Insert A, B, C, D: cache is now "B, C, D".
  • Lookup A: miss, single get, insert, purge, cache is now "C, D, A".
  • Lookup B: miss, singel get, insert, purge, cache is now "D, A, B".
Test Plan
  • Reduced cache size to 5, observed reasonable behavior on the array_slice() locally with bin/repository update + var_dump().
  • Used this script to estimate the size of 65535 cache entries as 7.5MB:
epriestley@orbital ~ $ cat size.php 
<?php

$cache = array();

$mem_start = memory_get_usage();
for ($ii = 0; $ii < 65535; $ii++) {
  $cache[sha1($ii)] = true;
}

echo number_format(memory_get_usage() - $mem_start)." bytes\n";
epriestley@orbital ~ $ php -f size.php 
7,602,176 bytes

Diff Detail

Repository
rP Phabricator
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

This revision is now accepted and ready to land.Feb 24 2017, 6:13 PM
This revision was automatically updated to reflect the committed changes.