Page MenuHomePhabricator

Possibly increase the width of the S3 -> Files -> Client pipeline
Changes PlannedPublic

Authored by epriestley on Jul 30 2018, 7:51 PM.
Tags
None
Referenced Files
F12839293: D19548.id46739.diff
Thu, Mar 28, 7:19 PM
Unknown Object (File)
Jan 28 2024, 8:56 AM
Unknown Object (File)
Dec 6 2023, 9:25 PM
Unknown Object (File)
Nov 30 2023, 4:19 AM
Unknown Object (File)
Nov 17 2023, 4:18 AM
Unknown Object (File)
Nov 16 2023, 4:52 PM
Unknown Object (File)
Nov 13 2023, 6:31 PM
Unknown Object (File)
Nov 11 2023, 2:45 PM
Subscribers
None

Details

Summary

Ref T12907. Our ability to serve files from S3 may be limited by HTTP latency for the S3 requests.

If so, we can improve throughput by making multiple simultaneous requests to S3 and buffering a few chunks ahead of the one we're processing.

Test Plan
  • With or without this patch, I max out my local uplink over wireless (which is about 80Mbps), so I can't really test that it's actually faster.
  • I technically have gigabit fiber onsite now, I just have to figure out which ethernet cable in my NOC (a stack of cardboard boxes in the garage) runs up to my office.
  • Otherwise, I can patch this to admin and pull locally.
  • I'll try to get one or more of these plans working and report back.

Diff Detail

Repository
rP Phabricator
Branch
filefuture1
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 20538
Build 27894: Run Core Tests
Build 27893: arc lint + arc unit

Event Timeline

I technically have gigabit fiber onsite now, I just have to figure out which ethernet cable in my NOC (a stack of cardboard boxes in the garage) runs up to my office.

I identified the cable which runs to my office (it's the dirty one with a bit of a blue smudge). The stack of boxes wasn't quite tall enough since the cable is quite short, but I found a piece of styrofoam I could use to extend the operational capabilities of my NOC.

However, I realize my laptop doesn't actually have an ethernet port, foiling this plan. I ordered a USB-C to Ethernet adapter that should arrive tomorrow.

I'll take this back until I get the adapter and have some evidence it actually works rather than just making everything wildly more complicated.