Page MenuHomePhabricator

Garbage collect old notifications
Closed, ResolvedPublic

Description

I think we should likely GC old notifications. While GC'ing feed stories is more arguable and I could go either way on it, notifications seem useless after a fairly short time window, and this table can get huge for active installs. We could do something like GC after 90 days safely, I think.

To do this, we'll need to add a chronologicalKey key to the table.

Even on this install, which is relatively small in terms of potential to generate these rows, we have 400K of them and 150MB of data:

mysql> show table status like 'feed_storynotification';
+------------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+------------+----------------+---------------------+-------------+------------+-------------+----------+----------------+---------+
| Name                   | Engine | Version | Row_format | Rows   | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free  | Auto_increment | Create_time         | Update_time | Check_time | Collation   | Checksum | Create_options | Comment |
+------------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+------------+----------------+---------------------+-------------+------------+-------------+----------+----------------+---------+
| feed_storynotification | InnoDB |      10 | Compact    | 397141 |            139 |    55279616 |               0 |    101302272 | 6262095872 |           NULL | 2015-06-08 17:51:12 | NULL        | NULL       | utf8mb4_bin |     NULL |                |         |
+------------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+------------+----------------+---------------------+-------------+------------+-------------+----------+----------------+---------+
1 row in set (0.09 sec)
$ ./bin/storage probe
Analyzing table sizes (this may take a moment)...
APPROXIMATE TABLE SIZES
...
phabricator_feed                    309.3 MB    4.3%
    feed_storyreference              70.3 MB    1.0%
    feed_storydata                   89.7 MB    1.3%
    feed_storynotification          149.3 MB    2.1%
...

This is the largest log-esque table that we don't currently GC.

Revisions and Commits

Restricted Differential Revision

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a project: Notifications.

On our install, this data is ~250MB.

joshuaspence added a revision: Restricted Differential Revision.Jun 10 2015, 12:16 PM
joshuaspence closed this task as Resolved by committing Restricted Diffusion Commit.Jun 10 2015, 8:55 PM
joshuaspence added a commit: Restricted Diffusion Commit.