Databend failed to start after removing access_key_id and secret_access_key .
[storage.s3]
bucket = "xx"
endpoint_url = "https://s3.ap-northeast-1.amazonaws.com"
#access_key_id = "xx"
#secret_access_key = "xx"
{"timestamp":"2022-12-12T16:44:44.716618Z","level":"WARN","fields":{"message":"load_via_profile_shared_credentials_file failed: No such file or directory (os error 2)","log.target":"reqsign::aws::config","log.module_path":"reqsign::aws::config","log.file":"/opt/rust/cargo/registry/src/github.com-1ecc6299db9ec823/reqsign-0.6.9/src/aws/config.rs","log.line":174},"target":"reqsign::aws::config"}
{"timestamp":"2022-12-12T16:44:44.716640Z","level":"WARN","fields":{"message":"load_via_profile_config_file failed: No such file or directory (os error 2)","log.target":"reqsign::aws::config","log.module_path":"reqsign::aws::config","log.file":"/opt/rust/cargo/registry/src/github.com-1ecc6299db9ec823/reqsign-0.6.9/src/aws/config.rs","log.line":176},"target":"reqsign::aws::config"}
{"timestamp":"2022-12-12T16:44:44.717063Z","level":"WARN","fields":{"message":"load credential via imds v2 failed: http://169.254.169.254/latest/api/token: status code 400","log.target":"reqsign::aws::credential","log.module_path":"reqsign::aws::credential","log.file":"/opt/rust/cargo/registry/src/github.com-1ecc6299db9ec823/reqsign-0.6.9/src/aws/credential.rs","log.line":141},"target":"reqsign::aws::credential"}
{"timestamp":"2022-12-12T16:44:44.717086Z","level":"INFO","fields":{"message":"load credential failed but we allowing anonymous access","log.target":"reqsign::aws::credential","log.module_path":"reqsign::aws::credential","log.file":"/opt/rust/cargo/registry/src/github.com-1ecc6299db9ec823/reqsign-0.6.9/src/aws/credential.rs","log.line":171},"target":"reqsign::aws::credential"}
{"timestamp":"2022-12-12T16:44:44.748011Z","level":"WARN","fields":{"message":"service=s3 operation=list path=/ -> errored: ObjectPermissionDenied (permanent) at ObjectPage::next_page => S3Error { code: \"AccessDenied\", message: \"Access Denied\", resource: \"\", request_id: \"xx\" }\n\nContext:\n response: Parts { status: 403, version: HTTP/1.1, headers: {\"x-amz-bucket-region\": \"ap-northeast-1\", \"x-amz-request-id\": \"xx\", \"x-amz-id-2\": \"xx\", \"content-type\": \"application/xml\", \"transfer-encoding\": \"chunked\", \"date\": \"Mon, 12 Dec 2022 16:44:44 GMT\", \"server\": \"AmazonS3\"} }\n service: s3\n path: /\n","log.target":"opendal::services","log.module_path":"opendal::layers::logging","log.file":"/opt/rust/cargo/registry/src/github.com-1ecc6299db9ec823/opendal-0.22.1/src/layers/logging.rs","log.line":1221},"target":"opendal::services"}
I found two problems:
x-aws-ec2-metadata-token-ttl-seconds request header for http://169.254.169.254/latest/api/token is required.
curl -X PUT -v "http://169.254.169.254/latest/api/token"
* Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> PUT /latest/api/token HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
< Content-Length: 0
< Date: Mon, 12 Dec 2022 16:23:52 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0
curl -X PUT -v "http://169.254.169.254/latest/api/token" -H "x-aws-ec2-metadata-token-ttl-seconds: 60"
* Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> PUT /latest/api/token HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.79.1
> Accept: */*
> x-aws-ec2-metadata-token-ttl-seconds: 60
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 60
< Content-Length: 56
< Date: Mon, 12 Dec 2022 16:25:20 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0
xxxxxxxxx
The response format for http://169.254.169.254/latest/meta-data/iam/security-credentials is json.
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 60"`
ROLE=`curl -H "x-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/`
curl -H "x-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLE
* Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/iam/security-credentials/xxxxxxxx HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.79.1
> Accept: */*
> x-aws-ec2-metadata-token: xxxxxxxx
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 60
< Content-Type: text/plain
< Accept-Ranges: none
< Last-Modified: Mon, 12 Dec 2022 15:42:45 GMT
< Content-Length: 1462
< Date: Mon, 12 Dec 2022 16:33:07 GMT
< Server: EC2ws
< Connection: close
<
{
"Code" : "Success",
"LastUpdated" : "2022-12-12T15:42:33Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "xxxxxxxx",
"SecretAccessKey" : "xxxxxxxx",
"Token" : "xxxxxxxx",
"Expiration" : "2022-12-12T22:12:07Z"
* Closing connection 0```
24 Dec 1, 2022
5 Jul 26, 2023
10 Dec 14, 2022
18 Oct 31, 2022
246 Dec 20, 2022
18 May 5, 2023
7.1k Jan 8, 2023
28 Jan 2, 2023
86 Dec 25, 2022
34 Nov 29, 2022
12 Sep 24, 2022
7 Oct 3, 2022
8 Sep 6, 2022
3 Jul 9, 2022
4 Apr 16, 2022
10 Jul 2, 2022
13 Dec 30, 2022
7 Sep 15, 2022
185 Dec 15, 2022