Page MenuHomePhabricator

Possibly increase the width of the S3 -> Files -> Client pipeline
Changes PlannedPublic

Authored by epriestley on Jul 30 2018, 7:51 PM.

Details

Summary

Ref T12907. Our ability to serve files from S3 may be limited by HTTP latency for the S3 requests.

If so, we can improve throughput by making multiple simultaneous requests to S3 and buffering a few chunks ahead of the one we're processing.

Test Plan
  • With or without this patch, I max out my local uplink over wireless (which is about 80Mbps), so I can't really test that it's actually faster.
  • I technically have gigabit fiber onsite now, I just have to figure out which ethernet cable in my NOC (a stack of cardboard boxes in the garage) runs up to my office.
  • Otherwise, I can patch this to admin and pull locally.
  • I'll try to get one or more of these plans working and report back.

Diff Detail

Repository
rP Phabricator
Branch
filefuture1
Lint
Lint OK
Unit
Unit Tests OK
Build Status
Buildable 20538
Build 27894: Run Core Tests
Build 27893: arc lint + arc unit

Event Timeline

epriestley created this revision.Jul 30 2018, 7:51 PM
epriestley requested review of this revision.Jul 30 2018, 7:52 PM

I technically have gigabit fiber onsite now, I just have to figure out which ethernet cable in my NOC (a stack of cardboard boxes in the garage) runs up to my office.

I identified the cable which runs to my office (it's the dirty one with a bit of a blue smudge). The stack of boxes wasn't quite tall enough since the cable is quite short, but I found a piece of styrofoam I could use to extend the operational capabilities of my NOC.

However, I realize my laptop doesn't actually have an ethernet port, foiling this plan. I ordered a USB-C to Ethernet adapter that should arrive tomorrow.

epriestley planned changes to this revision.Jul 30 2018, 11:14 PM

I'll take this back until I get the adapter and have some evidence it actually works rather than just making everything wildly more complicated.