Change Details

Users should have a reasonable way to export their data This is mostly problematic because the dumps may be very large, and we don't have very good infrastructure for handling multi-gigabyte files. Two possible approaches: - Use a daemon to upload things to S3, then serve from S3 directly. I don't love this because it involves a custom policy system (S3) on the most sensitive data. - Install a custom file storage engine which just points to backups on disk. When users want to export data, we create a new Files entry on their instance which points at the backup data, and then we could let them download it with a custom SSH handler. This is relatively clean technically (well, we have to proxy, so maybe not that clean) but likely to end up being very bizarre. Too much of this feels weird and we should probably do the S3 thing. - Ideal is probably that we improve large file support in Phabricator, then upload the data to their instance (presumably pushing it through to S3, ultimately) and then let them download it. This would use standard auth pathways without being too weird, but we need to get large file support on both the upload and download pathways for it to work. There's also a security concern: downloading the database backups will include session keys, CSRF tokens, any stored private keys, etc. We probably can't reasonably strip this data, although we could consider doing so in some cases. But "Export Data" needs to be tightly restricted.

Users should have a reasonable way to export their data. This is mostly problematic because the dumps may be very large, and we don't have very good infrastructure for handling multi-gigabyte files. Two possible approaches: - Use a daemon to upload things to S3, then serve from S3 directly. I don't love this because it involves a custom policy system (S3) on the most sensitive data. - Install a custom file storage engine which just points to backups on disk. When users want to export data, we create a new Files entry on their instance which points at the backup data, and then we could let them download it with a custom SSH handler. This is relatively clean technically (well, we have to proxy, so maybe not that clean) but likely to end up being very bizarre. Too much of this feels weird and we should probably do the S3 thing. - Ideal is probably that we improve large file support in Phabricator, then upload the data to their instance (presumably pushing it through to S3, ultimately) and then let them download it. This would use standard auth pathways without being too weird, but we need to get large file support on both the upload and download pathways for it to work. There's also a security concern: downloading the database backups will include session keys, CSRF tokens, any stored private keys, etc. We probably can't reasonably strip this data, although we could consider doing so in some cases. But "Export Data" needs to be tightly restricted.

Users should have a reasonable way to export their data. This is mostly problematic because the dumps may be very large, and we don't have very good infrastructure for handling multi-gigabyte files. Two possible approaches: - Use a daemon to upload things to S3, then serve from S3 directly. I don't love this because it involves a custom policy system (S3) on the most sensitive data. - Install a custom file storage engine which just points to backups on disk. When users want to export data, we create a new Files entry on their instance which points at the backup data, and then we could let them download it with a custom SSH handler. This is relatively clean technically (well, we have to proxy, so maybe not that clean) but likely to end up being very bizarre. Too much of this feels weird and we should probably do the S3 thing. - Ideal is probably that we improve large file support in Phabricator, then upload the data to their instance (presumably pushing it through to S3, ultimately) and then let them download it. This would use standard auth pathways without being too weird, but we need to get large file support on both the upload and download pathways for it to work. There's also a security concern: downloading the database backups will include session keys, CSRF tokens, any stored private keys, etc. We probably can't reasonably strip this data, although we could consider doing so in some cases. But "Export Data" needs to be tightly restricted.