Out of curiosity, why use Paste for that much code?

From T9566, these are probably 1MB JSON blobs from API calls.

But, yeah, can you just use P123 instead of {P123}? No one is really going to look at this content anyway, right? What about just uploading it as a file?

This is about Paste application index page. I guess more generally speaking it would paste search query result list.

Ohhh! Sorry, we usually use "preview" to mean "remarkup preview" (the comment box thing at the bottom of the page) and I figured you were putting {P123} in a comment and getting slow comment previews.

We should truncate total size on the list view, that's totally reasonable.

I tried implementing this myself but got a bit stuck. Since lines and line limit is passed to PhabricatorSourceCodeView from PhabricatorPasteSearchEngine::renderResultList() this looked like a good place to start, and pass line length limit. The problem I've encountered is that PhabricatorSourceCodeView receives lines already in HTML. Even though stripping tags and converting HTML entities back to original characters to get line length is easily doable but seems a bit awkward. And then truncating HTML markup is even more problematic...

There's a vaguely related but much less severe issue where the list view can take a long time to load because it needs to pull a bunch of files off S3 in order to generate the content caches.

I think the best general fix here is probably to add a "snippet" field to the Paste object and store a snippet on it, which we generate by doing something like taking the smaller of:

• the first 5 lines of the file; or
• the characters in the first 1,024 bytes of the file.

...for some values of "5" and "1024". I'm OK with these limits being fairly arbitrary since you can just click through anyway, and there is no reasonable use case for wanting 1MB lines on this screen.

We should maybe also store which type of truncation occurred (none, lines, bytes) so we can render an appropriate "..." on the following line, at the end of the last line, or omit it.

PhutilUTF8StringTruncator can perform safe, unicode-aware byte-limited text truncation.

One problem with this approach is that the migration is potentially huge, but not performing the migration would be annoying. We could populate the snippet lazily for a while, I guess, and eventually remove that code.

So I think the overall approach would be:

• Add a snippet field, or similar.
• Add a snippetType field, or similar, with values: no snippet, full, first lines (renders with "..." on next line), first bytes (renders with "..." at end of line).
• When saving a change to paste content, update the snippet content.
• When loading pastes, populate snippets for pastes with a "no snippet" value. Maybe get rid of this after some number of months.
• Generate the preview by highlighting the snippet instead of highlighting the full content.
• Remove any remaining code which causes a content load in any circumstances on the list view.

I haven't actually looked at the code and haven't touched Paste meaningfully in a while so some of that might be a little off.