Page MenuHomePhabricator

Phacility export process removes data from S3 which may still be referenced
Closed, ResolvedPublic

Description

The export process uses bin/files migrate to migrate file data to blob storage on the aux tier. As a side effect, unreferenced files have their source data destroyed.

For files in S3, this can destroy the live copy of the file data. If the export subsequently fails, this can destroy the only copy of the data.

To fix this, I plan to give bin/files migrate some sort of --idempotent flag to avoid removing data.

Event Timeline

epriestley closed this task as Resolved.Sep 6 2016, 8:59 PM

I cherry-picked those fixes to stable, created a test instance (test-on6dfrmgkt7t), uploaded a 100KB file which went to S3, exported the instance, and got a clean export with the file data retained on the instance.

I repeated the export process without issues, and verified the file still functioned correctly on the instance after multiple exports.