Great! I'm not sure how to use a trusted source IP, as CloudFront doesn't seem to provide any configuration in that regard, but just using the user-agent should not result in any reduced security, since no authenticated actions can be performed when this Host is set.

After doing some packet capture on my server to see exactly what's happening, I've found that when I configure CloudFront to forward the Host header (or All Headers), it will connect to my origin (code.simplyinsured.com), but the SSL handshake server name field will be set to d3vd7xwqgmo7c1.cloudfront.net. Since my server doesn't have a certificate for *.cloudfront.net, there's no way that's going to result in a valid SSL handshake.

However, Cloudfront no longer serves files on my domain, at all. I simply get a "CloudFront wasn't able to connect to the origin." error message. See P1843.

Yep!