Page MenuHomePhabricator

Standardize GarbageCollector TTLs
Closed, ResolvedPublic

Description

Currently, we have a bunch of GC TTL configuration settings:

  • gcdaemon.ttl.conduit-logs
  • gcdaemon.ttl.daemon-logs
  • gcdaemon.ttl.differential-parse-cache
  • gcdaemon.ttl.general-cache
  • gcdaemon.ttl.herald-transcripts
  • gcdaemon.ttl.markup-cache
  • gcdaemon.ttl.task-archive

These settings are rarely useful and pretty cluttery for new installs.

We actually have significantly more garbage collectors than this (about 20), but most of them don't even have TTL settings.

Having these settings is useful to some degree. I've heard of at least two reasonable use cases from installs that want to control TTLs for garbage collection:

  • One install has an information retention policy which requires purging the user activity logs after X days.
  • Another install wants to retain build logs for a very long time (perhaps indefinitely) so they can serve as a component of a long-lasting audit record.

I'd like to make these TTL settings part of PhabricatorGarbageCollector, merge them all into one dictionary, shove that dictionary out of the way, and probably put a CLI interface on it. Something like this:

$ ./bin/garbage status
| Collector  | TTL      |
|------------|----------|
| User Logs  | 180 days |
| Build Logs | 60 days  |
| ...        | ...      |

$ ./bin/garbage set-ttl user-logs --days 45
Changed TTL of user logs from 180 days to 45 days.
This change will take effect the next time the daemons are restarted.

$ ./bin/garbage set-ttl build-logs --never
Changed TTL of build logs from 60 days to indefinite retention.
This change will take effect the next time the daemons are restarted.

Then we get new settings "for free", the handful of installs that care about this stuff can configure it, and it's out of the way for the 99% of installs that don't care and are happy with the defaults.

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a project: Config.

wow man, thanks for thinking about this

After D14219, you can adjust GC policies with the new bin/garbage.

For example, to reduce the retention policy of the user activity log from 180 days to 90 days, you would run:

phabricator/ $ ./bin/garbage set-policy --collector user.logs --days 90

To prevent collection of, say, inbound mail, you would run:

phabricator/ $ ./bin/garbage set-policy --collector metamta.received --indefinite

You can use --default to restore an option to the default value. You can review collectors and their settings in ConfigGarbage Collectors.

After making an adjustment, restart the daemons for it to take effect.

NOTE: bin/garbage set-policy is just a user-friendly wrapper around writing to local config in conf/local/local.json, so you may need to pick up the updates it makes to that file if you're using some fancy deployment magic on top of Phabricator.