Page MenuHomePhabricator
Create Task
Maniphest T12377

S3 cannot migrate / upload files
Closed, WontfixPublic
Actions

Description

phabricator abff6dc8a9b4e081d74beaf8f062223cb5308ec3 (Wed, Mar 8)
arcanist 3b6b523c2b236e3724a1e115f126cb6fd05fa128 (Sat, Feb 18)
phutil 13a200ca7621ab2b48a0c395f52f8c4411bbc686 (Sat, Mar 4)

Configured as in guide:
./bin/config set amazon-s3.access-key QKYmzxcJOJrW5oma
./bin/config set amazon-s3.secret-key xkKTZ5W7Xift1WV3
./bin/config set storage.s3.bucket files.example.com
./bin/config set amazon-s3.region eu-central-1
./bin/config set amazon-s3.endpoint apigateway.eu-central-1.amazonaws.com
(do not worry, those are not real creds)
When migrating, i get this error:

phab@midgard:~/phab/phabricator$ ./bin/files migrate --engine amazon-s3 F16
F16: (74 KB) Migrating from "blob" to "amazon-s3"...
Failed! PhutilAWSException: AWS Request Failed<LF>HTTP Status Code: 403<LF>Response Body: <InvalidSignatureException><LF>  <Message>Credential should be scoped to correct service: 'apigateway'. </Message><LF></InvalidSignatureException><LF> in /home/phab/phab/libphutil/src/future/aws/PhutilAWSFuture.php:165<LF>Stack trace:<LF>#0 /home/phab/phab/libphutil/src/future/aws/PhutilAWSS3Future.php(63): PhutilAWSFuture->didReceiveResult(Array)<LF>#1 /home/phab/phab/libphutil/src/future/FutureProxy.php(58): PhutilAWSS3Future->didReceiveResult(Array)<LF>#2 /home/phab/phab/libphutil/src/future/FutureProxy.php(35): FutureProxy->getResult()<LF>#3 /home/phab/phab/phabricator/src/applications/files/engine/PhabricatorS3FileStorageEngine.php(80): FutureProxy->resolve()<LF>#4 /home/phab/phab/phabricator/src/applications/files/storage/PhabricatorFile.php(524): PhabricatorS3FileStorageEngine->writeFile('{"Description":...', Array)<LF>#5 /home/phab/phab/phabricator/src/applications/files/storage/PhabricatorFile.php(439): PhabricatorFile->writeToEngine(Object(PhabricatorS3FileStorageEngine), '{"Description":...', Array)<LF>#6 /home/phab/phab/phabricator/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php(169): PhabricatorFile->migrateToEngine(Object(PhabricatorS3FileStorageEngine), false)<LF>#7 /home/phab/phab/libphutil/src/parser/argument/PhutilArgumentParser.php(441): PhabricatorFilesManagementMigrateWorkflow->execute(Object(PhutilArgumentParser))<LF>#8 /home/phab/phab/libphutil/src/parser/argument/PhutilArgumentParser.php(333): PhutilArgumentParser->parseWorkflowsFull(Array)<LF>#9 /home/phab/phab/phabricator/scripts/files/manage_files.php(21): PhutilArgumentParser->parseWorkflows(Array)<LF>#10 {main}
[2017-03-09 16:23:20] EXCEPTION: (PhutilAWSException) AWS Request Failed
HTTP Status Code: 403
Response Body: <InvalidSignatureException>
  <Message>Credential should be scoped to correct service: 'apigateway'. </Message>
</InvalidSignatureException>
 at [<phutil>/src/future/aws/PhutilAWSFuture.php:165]
arcanist(head=master, ref.master=3b6b523c2b23), phabricator(head=master, ref.master=abff6dc8a9b4), phutil(head=master, ref.master=13a200ca7621)
  #0 PhutilAWSFuture::didReceiveResult(array) called at [<phutil>/src/future/aws/PhutilAWSS3Future.php:63]
  #1 PhutilAWSS3Future::didReceiveResult(array) called at [<phutil>/src/future/FutureProxy.php:58]
  #2 FutureProxy::getResult() called at [<phutil>/src/future/FutureProxy.php:35]
  #3 FutureProxy::resolve() called at [<phabricator>/src/applications/files/engine/PhabricatorS3FileStorageEngine.php:80]
  #4 PhabricatorS3FileStorageEngine::writeFile(string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:524]
  #5 PhabricatorFile::writeToEngine(PhabricatorS3FileStorageEngine, string, array) called at [<phabricator>/src/applications/files/storage/PhabricatorFile.php:439]
  #6 PhabricatorFile::migrateToEngine(PhabricatorS3FileStorageEngine, boolean) called at [<phabricator>/src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php:169]
  #7 PhabricatorFilesManagementMigrateWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:441]
  #8 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:333]
  #9 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/files/manage_files.php:21]

Related Objects

Event Timeline

rlex created this task.Mar 9 2017, 1:27 PM
rlex added a comment.Mar 9 2017, 1:29 PM

Probably worth mentioning that this creds are configured using IAM instead of root creds.
Policy for this user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::files.example.com",
                "arn:aws:s3:::files.example.com/*"
            ]
        }
    ]
}
rlex added a comment.Mar 9 2017, 1:37 PM

Ok, i'm an idiot.
I configured wrong endpoint.
It's working now.
Still, it might be good idea for mentioning this policy in install doc - it's not really obvious that you need

"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "arn:aws:s3:::*"

For IAM policy to work.

chad closed this task as Wontfix.Mar 9 2017, 2:45 PM
chad claimed this task.
Phabricator · Built by Phacility