Fixes T12661.
When changing the start date of an event from some time in the past to some time significantly in the future (more than 24 hours), we'd invalidate only future caches and leave users in an "away" state. Instead, just invalidate all past and future caches (this is simpler than trying to figure out a narrower window, and should not make us do too much extra work).
When uninviting users from events, their caches also didn't get cleared correctly. Instead, clear them.