I'm wondering how this should interact with T8407: Control diffusion's syntax highlighting.

It would be great if there was an ability for installations to write custom viewers for formats that are either more obscure (MIF Framemaker format), or even in house formats (.myformat)

(This still needs documentation and validation for the configuration option before it's done, but I don't have any short-term plans to pursue those.)

To customize this:

Safari, Chrome, and Firefox all seem fine with PNG icons so I think we don't need to deal with actual .ico icons.

Also relevant:

(For completeness, this came up via the support queue in PHI411.)

These changes are all deployed here, now. The embed element only got touched lightly but is at least slightly better. See T4340 for further adventures in Content-Security-Policy.

• "Download" is a form, so you can't command-click it.
• The whole thing is a <div href="..." /> (huh?) so you can't command-click it to open it in a new window.
• When you click it for a non-image file, you get this weird interstitial that you can leave comments on if you click an additional button, which uses janky animations and AJAX. This feature is pretty half-baked and I've never seen anyone actually use it. It's possibly a net negative in its current form.
• There is no way to actually show the text file in the browser! ARHGRH

Okay, here's another one of these:

FWIW, I am using a custom file and mail engine that utilises the AWS SDK and supports instance profile credentials. See P2082 and P2083. You need to install and require the AWS SDK before using these extensions.

T10604 and T4369 do not impact LFS (broadly, they apply only to POST requests, and LFS uses PUT to upload data).

We have an outstanding support issue (PHI204) about timeouts with large files.

Per above, pushes to staging areas did not push LFS in the past. We should figure out what the state of the world is now and try to resolve it in the upstream, and document anything we can't work around or resolve.

@Grimeh, above, reports that this doesn't work over HTTP. I believe it does, but we should verify this.

I don't see this increasing storage usage - just shifting it. Right now because Phacility doesn't support LFS, all those large files are going straight into Git, which not only ends up on an EBS SSD, but also has to be cloned every single time which increases bandwidth usage. At least with LFS those costs are going to be from S3 (which I believe is cheaper than EBS SSD per GB), and bandwidth would be down as new clones no longer need to pull every version of a large file in the Git history.

I think the current state of things is:

@epriestley I've pinged you via pm about prioritization.

Per above, not planning to actually go forward with the GC step since the impact isn't ultimately very large.

Looking at the actual data, I'm less sure this is actually a good strategy. Here's the data for this install, considering the production configuration of storage.mysql-engine.max-size as 65535:

See Planning for information on planning and timelines.

Thanks for the detailed response, I certainly didn't expect it.

I believe the location of the OOM is a little misleading -- the real problem is loadFileData(). This returns the entire file as a string, and will thus:

Yeah this feels low value vs. building Dropbox. Closing in favor of overengineering.

I wonder if this OOM error can also be hit in other workflow though, given that it seems to occur in PhabricatorFileStorageEngine::getRawFileDataIterator.

Yep, agreed that the re-targeted proposal is a better solution... I had just assumed that this was a documentation oversight.

I retargeted this; I think the proposed solution is not the best solution we can find to the problem.

We should probably just remove this workflow, it's not clear to me that there's ever any reason for anyone to run bin/files purge in the modern codebase.

Just keeping an eye on this: growth on these shards is slow and both are still below the 80% caution threshold, so I'm punting this for another week.

We appear to have survived this.

Only db001 and db002 are anywhere close to having problems with this and they still have a large amount of headroom, and we can bin/storage optimize at least some amount of additional headroom into existence, so this isn't "we're about to fall off a cliff" urgent or anything, but would be nice to tackle sooner than later.

A customer is hitting an issue where pushes to staging areas do not push LFS objects. I'd guess this is an LFS upstream problem based on a hazy recollection of events here.

FWIW we have seen several users attempting to distribute l33t w4r3z via Wikimedia's instance of Phabricator. I had to set file upload limits to < 8MB in order to prevent chunked file storage.

makes sense. thanks!

There may also be a similar attack possible by adding address in local network space as a JIRA install, outbound HTTP hook (unlikely a problem today, but maybe in the future), Phabricator OAuth install, LDAP server, etc. Today, we're generally balancing things as "by default, it's OK for administrators to register services in local network space".

Harbormaster and Diffusion do not use the outbound blacklist, since it would prevent users from interacting with local-network build hosts (for example, a local install of Jenkins) or local-network repositories (for example, a local install of GitHub Enterprise). We currently consider these use cases to be common / valuable.

Shouldn't this behavior be blocked by the default settings of security.outbound-blacklist? 169.254.0.0/16 appears in that list, but as you say, a Harbormaster fetch to http://169.254.169.254 succeeds as expected

No. It's not that the builds can access credentials on the build hosts -- that part is a general problem -- it's that the Phabricator application hosts can be made to divulge their credentials (which are necessarily the same as the S3 credentials, because the same hosts must access S3) by instructing them to run builds that use 169.254.169.254 as a "build server".

In T5155#223416, @epriestley wrote:

As a general note here, we've been vulnerable to credential theft from the local service throughout this discussion, and still are until T12701 resolves: attackers can create a Harbormaster build plan which sends requests to 169.254.169.254, then read credentials from the output of the "failed build".

Users can also currently add http://169.254.169.254/ as a Git repository, although I'm not certain if they can actually capture credentials by doing this.

As a general note here, we've been vulnerable to credential theft from the local service throughout this discussion, and still are until T12701 resolves: attackers can create a Harbormaster build plan which sends requests to 169.254.169.254, then read credentials from the output of the "failed build".

We run into this as well, mainly when uploading photos of flipover sheets and whiteboards (to Maniphest or Phriction). Our workaround for now is to ⌘ + click, so the image will render directly into the browser instead of using the lightbox.

Maybe "duration of all file upload requests" / "duration of all requests". That isn't a perfect metric but should be a reasonable-ish proxy for real resources we care about, I think.

What would the appropriate query be? "Number of uploaded files with size greater than X as a percentage of all files/file bytes"?

nice progress bar

I suspect this isn't worth the complexity. If you drag-and-drop, the maximum request size is 4MB (one chunk) and we get a nice progress bar, resumable uploads, and the bucket can remain completely private. And we can do encryption and deduplicate file blocks.

We'd also have to add locking to prevent a race like this:

My gut is to wait for users to hit those cases before building solutions for them since I think it's unlikely any of that will ever turn into a real problem.

@epriestley should it be possible to un-delete a File, the same way it's possible to archive and then un-archive a Paste? The only benefit I can see is that in an emergency where someone accidentally kicked off a delete for a bunch of files they didn't intend to, someone could stop the daemons and perform an undelete operation before restarting the daemons (instead of flushing the worker queue). I actually mostly coded it this way already by following along with PhabricatorPasteStatusTransaction.