Ok, i'm an idiot.
I configured wrong endpoint.
It's working now.
Still, it might be good idea for mentioning this policy in install doc - it's not really obvious that you need
"Effect": "Allow", "Action": "s3:ListAllMyBuckets", "Resource": "arn:aws:s3:::*"
For IAM policy to work.