[Bug]: Authentication failure on v5.24.0 - incorrectly trying to use ec2 IMDS #34234

adamaziz15 · 2023-11-02T23:59:55Z

Terraform Core Version

1.5.3

AWS Provider Version

5.24.0

Affected Resource(s)

No response

Expected Behavior

I have a provider configuration just like this:

provider "aws" {
    region = "us-east-1"
    shared_credentials_files = ["path/to/my/shared_creds_file"]

    assume_role {
        role_arn = "my-assumed-role-arn"
        session_name = "some-session-name"
    }
}

Expected Behavior: The terraform provider should be able to authenticate without any issues.

Actual Behavior

Instead of authenticating properly I get the following error:

Error: no EC2 IMDS role found

It's my understanding that ec2 imds is only to be used when running TF from an ec2 instance and trying to use the instance metadata to authenticate. So in this case I think maybe the wrong authentication strategy is being used. This works fine on v5.23.1

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

The provider configuration is very similar to this:

provider "aws" {
    region = "us-east-1"
    shared_credentials_files = ["path/to/my/shared_creds_file"]

    assume_role {
        role_arn = "my-assumed-role-arn"
        session_name = "some-session-name"
    }
}

I have a creds file at the shared_credentials_files path I provided and it looks like:

[default]
aws_access_key_id=<my_aws_access_key>
aws_secret_access_key=<my_aws_secret_access_key>

Steps to Reproduce

use a shared credentials file for authentication
use terraform-provider-aws v5.24.0
run terraform plan 💥

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

The text was updated successfully, but these errors were encountered:

github-actions · 2023-11-03T00:00:05Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

ewbankkit · 2023-11-03T12:30:20Z

Potential cause: aws/aws-sdk-go-v2#2328.

yogeek · 2023-11-03T13:26:32Z

Same issue here.
The same action on AWS is :

successful with AWS CLI from local machine
successful with terraform from local machine
failed with terraform from EC2 instance

I tried to set the AWS_EC2_METADATA_V1_DISABLED=false before launching the terraform plan as mentionned in the linked issue above, but it fails anyway...

It is blocking all our infra pipelines so forced to downgrade for now

gdavison · 2023-11-03T19:22:53Z

Hi @adamaziz15 and @yogeek, thanks for reporting this. Unfortunately, the IMDS error message is a bit of a red herring. The AWS SDK for Go v2 always tries IMDS unless it is explicitly disabled using the provider configuration parameter skip_metadata_api_check or the environment variable AWS_EC2_METADATA_DISABLED (Note that this is different from AWS_EC2_METADATA_V1_DISABLED). We will address the misleading error message in hashicorp/aws-sdk-go-base#747.

Could either of you include a debug log of the authentication flow, please? Set the environment variable TF_LOG=DEBUG. More information on debugging Terraform can be found at https://developer.hashicorp.com/terraform/internals/debugging

@yogeek, I see that you're running on an EC2 instance. Is that within a container?

@adamaziz15, are you running on EC2?

yogeek · 2023-11-03T19:40:57Z

@gdavison yes terraform is running inside a container in an EC2 instance

gdavison · 2023-11-03T20:31:54Z

Do either of your shared configuration or credentials files contain any leading whitespace? There's a change in how the AWS SDK is parsing the file which causes it to fail silently if there is leading whitespace: aws/aws-sdk-go-v2#2349

gdavison · 2023-11-03T20:38:18Z

@yogeek do you have a hop limit of 2 set on the underlying EC2 instance? It's required with IMDS v2 (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html#imds-considerations)

adamaziz15 · 2023-11-04T03:20:36Z

Hi @adamaziz15 and @yogeek, thanks for reporting this. Unfortunately, the IMDS error message is a bit of a red herring. The AWS SDK for Go v2 always tries IMDS unless it is explicitly disabled using the provider configuration parameter skip_metadata_api_check or the environment variable AWS_EC2_METADATA_DISABLED (Note that this is different from AWS_EC2_METADATA_V1_DISABLED). We will address the misleading error message in hashicorp/aws-sdk-go-base#747.

Could either of you include a debug log of the authentication flow, please? Set the environment variable TF_LOG=DEBUG. More information on debugging Terraform can be found at https://developer.hashicorp.com/terraform/internals/debugging

@yogeek, I see that you're running on an EC2 instance. Is that within a container?

@adamaziz15, are you running on EC2?

@gdavison I'm not running on EC2. I may have whitespace in the file though as I'm writing it programmatically so definitely something I should check there.

Here's the debug log from terraform:

2023-11-04T03:12:42.310Z [DEBUG] Starting graph walk: walkPlan
2023-11-04T03:12:42.312Z [DEBUG] created provider logger: level=debug
2023-11-04T03:12:42.312Z [INFO]  provider: configuring client automatic mTLS
2023-11-04T03:12:42.320Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5]
2023-11-04T03:12:42.321Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=359
2023-11-04T03:12:42.321Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5
2023-11-04T03:12:42.433Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: configuring server automatic mTLS: timestamp=2023-11-04T03:12:42.433Z
2023-11-04T03:12:42.454Z [DEBUG] provider: using plugin: version=5
2023-11-04T03:12:42.455Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: plugin address: address=/tmp/plugin108359113 network=unix timestamp=2023-11-04T03:12:42.454Z
2023-11-04T03:12:43.525Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: assume_role configuration set: @module=aws tf_aws.assume_role.role_arn=arn:aws:iam::564782862755:role/app-3704f91c-8b1d-4 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ConfigureProvider @caller=github.com/hashicorp/terraform-provider-aws/internal/provider/provider.go:505 tf_aws.assume_role.external_id= tf_aws.assume_role.session_name=co-managed-564782862755 tf_aws.assume_role.source_identity= tf_mux_provider=*schema.GRPCProviderServer tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 timestamp=2023-11-04T03:12:43.525Z
2023-11-04T03:12:43.526Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Configuring Terraform AWS Provider: tf_rpc=ConfigureProvider @caller=github.com/hashicorp/terraform-provider-aws/internal/conns/config.go:125 tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 @module=aws timestamp=2023-11-04T03:12:43.525Z
2023-11-04T03:12:43.526Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Resolving credentials provider: tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer timestamp=2023-11-04T03:12:43.526Z
2023-11-04T03:12:43.526Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Loading configuration: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider tf_mux_provider=*schema.GRPCProviderServer timestamp=2023-11-04T03:12:43.526Z
2023-11-04T03:12:43.527Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Retrieving credentials: tf_mux_provider=*schema.GRPCProviderServer tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp=2023-11-04T03:12:43.526Z
2023-11-04T03:12:43.527Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: aws.region= rpc.service= tf_mux_provider=*schema.GRPCProviderServer @module=aws.aws-base http.request.body= http.request.header.amz_sdk_request="attempt=1; max=3" http.request.header.x_aws_ec2_metadata_token_ttl_seconds=300 net.peer.name=169.254.169.254 @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.5.3 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 ft/ec2-imds" rpc.method= tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider http.method=PUT http.url=http://169.254.169.254/latest/api/token rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 timestamp=2023-11-04T03:12:43.527Z
2023-11-04T03:12:43.531Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 aws.region= tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ConfigureProvider @module=aws.aws-base http.response.body="Not Found
" http.response.header.content_type="text/plain; charset=utf-8" http.response.header.date="Sat, 04 Nov 2023 03:12:43 GMT" http.status_code=404 rpc.method= rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 http.duration=3 http.response.header.x_content_type_options=nosniff http.response_content_length=10 rpc.service= tf_mux_provider=*schema.GRPCProviderServer timestamp=2023-11-04T03:12:43.531Z
2023-11-04T03:12:43.531Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: request failed with unretryable error http response error StatusCode: 404, request to EC2 IMDS failed: tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider @module=aws.aws-base tf_aws.sdk=aws-sdk-go-v2 rpc.system=aws-api tf_mux_provider=*schema.GRPCProviderServer aws.region= rpc.service= rpc.method= @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-04T03:12:43.531Z
2023-11-04T03:12:43.531Z [WARN]  provider.terraform-provider-aws_v5.24.0_x5: falling back to IMDSv1: operation error ec2imds: getToken, http response error StatusCode: 404, request to EC2 IMDS failed: rpc.service= rpc.system=aws-api tf_provider_addr=registry.terraform.io/hashicorp/aws rpc.method= tf_aws.sdk=aws-sdk-go-v2 tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 tf_rpc=ConfigureProvider aws.region= @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:31 timestamp=2023-11-04T03:12:43.531Z
2023-11-04T03:12:43.532Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: @module=aws.aws-base http.request.header.amz_sdk_request="attempt=1; max=3" net.peer.name=169.254.169.254 @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.request.body= rpc.service= tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ConfigureProvider aws.region= http.method=GET http.url=http://169.254.169.254/latest/meta-data/iam/security-credentials/ http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.5.3 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 ft/ec2-imds" rpc.method= rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 timestamp=2023-11-04T03:12:43.532Z
2023-11-04T03:12:43.533Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: http.response.body="Not Found
" http.response.header.x_content_type_options=nosniff tf_provider_addr=registry.terraform.io/hashicorp/aws tf_aws.sdk=aws-sdk-go-v2 tf_mux_provider=*schema.GRPCProviderServer @module=aws.aws-base aws.region= rpc.method= http.response_content_length=10 http.status_code=404 rpc.service= @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.duration=0 http.response.header.content_type="text/plain; charset=utf-8" tf_rpc=ConfigureProvider http.response.header.date="Sat, 04 Nov 2023 03:12:43 GMT" rpc.system=aws-api tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 timestamp=2023-11-04T03:12:43.533Z
2023-11-04T03:12:43.533Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: request failed with unretryable error http response error StatusCode: 404, request to EC2 IMDS failed: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base rpc.service= tf_provider_addr=registry.terraform.io/hashicorp/aws aws.region= tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ConfigureProvider tf_aws.sdk=aws-sdk-go-v2 rpc.method= rpc.system=aws-api tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 timestamp=2023-11-04T03:12:43.533Z
2023-11-04T03:12:43.533Z [ERROR] provider.terraform-provider-aws_v5.24.0_x5: Response contains error diagnostic: tf_req_id=13195a74-49d8-65ad-df49-426b685c16b3 @module=sdk.proto diagnostic_detail="Please see https://registry.terraform.io/providers/hashicorp/aws
for more information about providing credentials.

Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, http response error StatusCode: 404, request to EC2 IMDS failed
" diagnostic_severity=ERROR tf_proto_version=5.4 tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_summary="No valid credential sources found" tf_rpc=Configure timestamp=2023-11-04T03:12:43.533Z
2023-11-04T03:12:43.534Z [ERROR] vertex "provider[\"registry.terraform.io/hashicorp/aws\"]" error: No valid credential sources found
2023-11-04T03:12:43.537Z [INFO]  backend/local: plan operation completed

Changes to Outputs:
  - js-int-tests-with-backend_cd16aab9b7a3-next-demo-aws_cloudfront_id = "E1F5NFJ11655Q4" -> null

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
╷
│ Error: No valid credential sources found
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on main.tf line 3, in provider "aws":
│    3: provider "aws" {
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, http response error StatusCode: 404, request to EC2 IMDS failed
│ 
╵
2023-11-04T03:12:43.796Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2023-11-04T03:12:43.806Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=359
2023-11-04T03:12:43.806Z [DEBUG] provider: plugin exited
2023-11-04 03:12:43,826 | INFO (terraform.terraform:463) | TF RUN (<TerraformConfig for <Application 7: Next App AWS> @ /tmp/3704f91c-8b1d-4596-a5af-7fb4c603172b/amazon-ecs/3>): [['terraform', 'plan']] 1 in 6 seconds

There is more from before those logs, but its seems to be mostly noise, like the following:

2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_appautoscaling_policy.cd16aab9b7a3-backend-main-45735a_down" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_s3_bucket.cf_logs" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_security_group.rds_sg" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_s3_bucket_acl.cf_logs_bucket_acl" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_acm_certificate.cd16aab9b7a3-next-demo-aws_cert" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_lb_listener_certificate.cd16aab9b7a3-next-demo-aws_main" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_lb_listener_certificate.cd16aab9b7a3-next-demo-aws_update-subscribe-button" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_ecr_repository.cd16aab9b7a3-backend-update-sub-54f2cc_repo" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "data.aws_iam_policy_document.ecs_events_role" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "data.aws_iam_policy_document.ecs_execution_role" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "data.aws_iam_policy_document.ecs_service_policy" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_acm_certificate_validation.main_cd16aab9b7a3-next-demo-aws_cert" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-04T03:12:40.670Z [DEBUG] ProviderTransformer: "aws_cloudwatch_event_target.cd16aab9b7a3-next-demo-aws_ecs_events" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]

adamaziz15 · 2023-11-04T03:25:52Z

@gdavison Interesting enough, I still get the same error:
Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, access disabled to EC2 IMDS via client option, or "AWS_EC2_METADATA_DISABLED" environment variable

This is with the provider configuration option:

provider "aws" {
    skip_metadata_api_check = true
    ...
}

yogeek · 2023-11-06T11:59:44Z

@gdavison my situation seems different than the one of @adamaziz15

Context :

we use EC2 as jenkins worker (jenkins EC2 plugin), this EC2 has an instance profile that is allowed to assume roles in different target accounts
we have a pipeline that
- set AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE to specific location to be sure it is only used by the current pipeline workspace

AWS_CONFIG_FILE = "${WORKSPACE}/.aws_config"
AWS_SHARED_CREDENTIALS_FILE = "${WORKSPACE}/.aws_credentials"

configures aws_config file to set up different profiles, one of which is called target and uses the EC2 instance metadata to do an assume role in another account like this

[profile target]
                        region = eu-central-1
                        credential_source = Ec2InstanceMetadata
                        role_arn = arn:aws:iam::XXXYYYZZZ:role/my_role_path/demo_deployment
                        external_id = xxxxxxxxxxxxxxxxx

run a docker container with terraform in it (mounting all the needed locations and vars for AWS env to be correctly configured in the container)
launches a terraform code with AWS provider configured to use profile = "target" and that does a datasource to get a VPC by tags.

Here is the debug log :

Beggining of logs... (collapsed to ease reading)

bash-5.1# TF_LOG=DEBUG terraform plan
2023-11-06T11:30:53.084Z [INFO]  Terraform version: 1.1.9
2023-11-06T11:30:53.084Z [INFO]  Go runtime version: go1.17.2
2023-11-06T11:30:53.084Z [INFO]  CLI args: []string{"terraform", "plan"}
[2023-11-06T11:30:53.084Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2023-11-06T11:30:53.084Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2023-11-06T11:30:53.084Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2023-11-06T11:30:53.084Z [DEBUG] ignoring non-existing provider search directory /root/.terraform.d/plugins
2023-11-06T11:30:53.084Z [DEBUG] ignoring non-existing provider search directory /root/.local/share/terraform/plugins
2023-11-06T11:30:53.084Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2023-11-06T11:30:53.084Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2023-11-06T11:30:53.085Z [INFO]  CLI command args: []string{"plan"}
2023-11-06T11:30:53.085Z [DEBUG] New state was assigned lineage "e938eece-c4d7-5ff8-c451-f5751e0a8450"
2023-11-06T11:30:54.052Z [DEBUG] checking for provisioner in "."
2023-11-06T11:30:54.052Z [DEBUG] checking for provisioner in "/usr/local/bin"
2023-11-06T11:30:54.053Z [INFO]  backend/local: starting Plan operation
2023-11-06T11:30:54.053Z [DEBUG] created provider logger: level=debug
2023-11-06T11:30:54.053Z [INFO]  provider: configuring client automatic mTLS
2023-11-06T11:30:54.062Z [DEBUG] provider: starting plugin: ]path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5]
2023-11-06T11:30:54.062Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=101
2023-11-06T11:30:54.062Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5
2023-11-06T11:30:54.135Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: configuring server automatic mTLS: timestamp=2023-11-06T11:30:54.135Z
2023-11-06T11:30:54.153Z [DEBUG] provider: using plugin: version=5
2023-11-06T11:30:54.153Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: plugin address: address=/tmp/plugin1151903341 network=unix timestamp=2023-11-06T11:30:54.153Z
2023-11-06T11:30:54.758Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-11-06T11:30:54.763Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=101
2023-11-06T11:30:54.763Z [DEBUG] provider: plugin exited
2023-11-06T11:30:54.763Z [DEBUG] Building and walking validate graph
2023-11-06T11:30:54.763Z [DEBUG] ProviderTransformer: "data.aws_caller_identity.target_account" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-06T11:30:54.763Z [DEBUG] ProviderTransformer: "data.aws_vpc.k8s_vpc" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-06T11:30:54.763Z [DEBUG] ReferenceTransformer: "data.aws_vpc.k8s_vpc" references: []
2023-11-06T11:30:54.763Z [DEBUG] ReferenceTransformer: "output.vpc" references: [data.aws_vpc.k8s_vpc]
2023-11-06T11:30:54.763Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/aws\"]" references: []
2023-11-06T11:30:54.763Z [DEBUG] ReferenceTransformer: "data.aws_caller_identity.target_account" references: []
2023-11-06T11:30:54.764Z [DEBUG] Starting graph walk: walkValidate
2023-11-06T11:30:54.764Z [DEBUG] created provider logger: level=debug
2023-11-06T11:30:54.764Z [INFO]  provider: configuring client automatic mTLS
2023-11-06T11:30:54.772Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5]
2023-11-06T11:30:54.772Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=115
2023-11-06T11:30:54.772Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5
2023-11-06T11:30:54.846Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: configuring server automatic mTLS: timestamp=2023-11-06T11:30:54.846Z
2023-11-06T11:30:54.865Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: plugin address: address=/tmp/plugin2854219547 network=unix timestamp=2023-11-06T11:30:54.865Z
2023-11-06T11:30:54.865Z [DEBUG] provider: using plugin: version=5
2023-11-06T11:30:55.403Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-11-06T11:30:55.409Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=115
2023-11-06T11:30:55.409Z [DEBUG] provider: plugin exited
2023-11-06T11:30:55.409Z [INFO]  backend/local: plan calling Plan
2023-11-06T11:30:55.409Z [DEBUG] Building and walking plan graph for NormalMode
2023-11-06T11:30:55.409Z [DEBUG] ProviderTransformer: "data.aws_vpc.k8s_vpc (expand)" (*terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-06T11:30:55.409Z [DEBUG] ProviderTransformer: "data.aws_caller_identity.target_account (expand)" (*terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-11-06T11:30:55.409Z [DEBUG] ReferenceTransformer: "data.aws_caller_identity.target_account (expand)" references: []
2023-11-06T11:30:55.409Z [DEBUG] ReferenceTransformer: "data.aws_vpc.k8s_vpc (expand)" references: []
2023-11-06T11:30:55.409Z [DEBUG] ReferenceTransformer: "output.vpc" references: [data.aws_vpc.k8s_vpc (expand)]
2023-11-06T11:30:55.410Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/aws\"]" references: []
2023-11-06T11:30:55.410Z [DEBUG] Starting graph walk: walkPlan
2023-11-06T11:30:55.410Z [DEBUG] created provider logger: level=debug
2023-11-06T11:30:55.410Z [INFO]  provider: configuring client automatic mTLS
2023-11-06T11:30:55.418Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5]
2023-11-06T11:30:55.418Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=128
2023-11-06T11:30:55.418Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5
2023-11-06T11:30:55.498Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: configuring server automatic mTLS: timestamp=2023-11-06T11:30:55.498Z
2023-11-06T11:30:55.517Z [DEBUG] provider: using plugin: version=5
2023-11-06T11:30:55.517Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: plugin address: address=/tmp/plugin3124861323 network=unix timestamp=2023-11-06T11:30:55.517Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Configuring Terraform AWS Provider: @caller=github.com/hashicorp/terraform-provider-aws/internal/conns/config.go:125 tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws @module=aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Resolving credentials provider: @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Using profile: tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider @module=aws.aws-base tf_aws.profile=target tf_aws.profile.source=provider tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Using shared credentials files: @module=aws.aws-base tf_aws.shared_credentials_files.source=envvar tf_rpc=ConfigureProvider tf_aws.shared_credentials_files=[/app/toto/.aws_credentials] tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Using shared configuration files: tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_aws.shared_config_files=[/app/toto/.aws_config] tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_aws.shared_config_files.source=envvar timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.035Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Loading profile: @module=aws.aws-base tf_aws.profile=target tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.035Z
2023-11-06T11:30:56.037Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Setting profile: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 tf_aws.profile=target tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @module=aws.aws-base tf_aws.profile.source=provider tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.037Z
2023-11-06T11:30:56.037Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Loading configuration: tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_mux_provider=*schema.GRPCProviderServer timestamp=2023-11-06T11:30:56.037Z
2023-11-06T11:30:56.039Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Retrieving credentials: tf_mux_provider=*schema.GRPCProviderServer @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 @module=aws.aws-base tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.039Z
2023-11-06T11:30:56.039Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 aws.region= http.request.body= rpc.system=aws-api tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ConfigureProvider http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 ft/ec2-imds" net.peer.name=169.254.169.254 rpc.method= tf_aws.sdk=aws-sdk-go-v2 @module=aws.aws-base http.method=PUT http.request.header.x_aws_ec2_metadata_token_ttl_seconds=300 tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 rpc.service= tf_provider_addr=registry.terraform.io/hashicorp/aws http.request.header.amz_sdk_request="attempt=1; max=3" http.url=http://169.254.169.254/latest/api/token timestamp=2023-11-06T11:30:56.039Z
2023-11-06T11:30:56.040Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: http.response.header.x_aws_ec2_metadata_token_ttl_seconds=300 rpc.service= rpc.system=aws-api tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.response.body="AQAEACZFnc2Y7ZVy7qcKLqkEszHahmf9TVW_LBEhyrfgxKFP6obTaQ==
" http.response.header.date="Mon, 06 Nov 2023 11:30:56 GMT" @module=aws.aws-base aws.region= http.duration=0 http.response.header.content_type=text/plain http.response.header.server=EC2ws http.response_content_length=56 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider http.status_code=200 rpc.method= tf_aws.sdk=aws-sdk-go-v2 timestamp=2023-11-06T11:30:56.040Z
2023-11-06T11:30:56.040Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: rpc.system=aws-api tf_mux_provider=*schema.GRPCProviderServer @module=aws.aws-base http.request.body= http.request.header.amz_sdk_request="attempt=1; max=3" tf_aws.sdk=aws-sdk-go-v2 tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider http.url=http://169.254.169.254/latest/meta-data/iam/security-credentials/ http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 ft/ec2-imds" net.peer.name=169.254.169.254 rpc.service= aws.region= http.method=GET rpc.method= tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.request.header.x_aws_ec2_metadata_token=AQAEACZFnc2Y7ZVy7qcKLqkEszHahmf9TVW_LBEhyrfgxKFP6obTaQ== timestamp=2023-11-06T11:30:56.040Z
2023-11-06T11:30:56.041Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.response.header.content_type=text/plain http.response.header.date="Mon, 06 Nov 2023 11:30:56 GMT" http.response.header.x_aws_ec2_metadata_token_ttl_seconds=300 tf_rpc=ConfigureProvider @module=aws.aws-base http.response.header.accept_ranges=none rpc.service= tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_aws.sdk=aws-sdk-go-v2 tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 http.response.body="gtp-jenkins-slaves-instance-profile
" http.response.header.server=EC2ws http.response_content_length=35 aws.region= http.duration=0 http.response.header.last_modified="Mon, 06 Nov 2023 11:10:28 GMT" http.status_code=200 rpc.method= rpc.system=aws-api timestamp=2023-11-06T11:30:56.041Z
2023-11-06T11:30:56.041Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: http.request.header.x_aws_ec2_metadata_token=AQAEACZFnc2Y7ZVy7qcKLqkEszHahmf9TVW_LBEhyrfgxKFP6obTaQ== tf_aws.sdk=aws-sdk-go-v2 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @module=aws.aws-base aws.region= http.request.header.amz_sdk_request="attempt=1; max=3" http.url=http://169.254.169.254/latest/meta-data/iam/security-credentials/gtp-jenkins-slaves-instance-profile http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 ft/ec2-imds" rpc.method= rpc.service= tf_provider_addr=registry.terraform.io/hashicorp/aws http.method=GET http.request.body= net.peer.name=169.254.169.254 rpc.system=aws-api tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-06T11:30:56.041Z
2023-11-06T11:30:56.042Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: http.duration=0 http.response.header.accept_ranges=none http.response.header.content_type=text/plain http.response.header.x_aws_ec2_metadata_token_ttl_seconds=300 rpc.method= rpc.service= @module=aws.aws-base http.response.body="{
  "Code" : "Success",
  "LastUpdated" : "2023-11-06T11:11:01Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIA************GJMD",
  "SecretAccessKey" : "V/XE********************************t5fF",
  "Token" : "IQoJb3JpZ2luX2VjEFMaDGV1LWNlbnRyYWwtMSJIMEYCIQDn/rsy7BfxLNgE+L98InVLDvUV3wWk4z8tchjNPcIuJAIhAMMyhUdo2iHzX/MDzeBOcmEIJjusvGAkw7QNvy+R0SQCKssFCIz//////////wEQBBoMNDA1MjIzODQ4NTUwIgzANjTu8ljG2q7D23gqnwXoHNRCoX9e443vi9+0hXcGq+a3oRMnJHaa3vO72Hw1cQ4JEwTfY7XUReTGmdeAhdLJtR3kSC5EN63sivomFzmF49P3TaLIQg4j2+HYafIFLdQHjJpkZL4e7BanHWchgFWdfFGuNoy9nzR/Tn9l+zXSb8gxYpg//+bMv/wJD34kWISXsPokQaqHnIx+2+msBKzU1AxUrEF/NeEU0iKScLZaiApZSdLN82lv2CZBw8WCPWsEoEr2dMk/SzkusnzngjSUQCkhNcY7ofhDTiOCWco4PGFhB5MEO7Sk63jLyO9qsg9fd0wdLuChurlSSa8Ff2LxUU2VGr0BxpBH0oXgvMTzdYiqFutNKR29jCvxpsbMLmD4hwn6pFgXdNufot+/khxO3BNePrGtRAmv6hx7VJSB91A+DCo75hGRvs+RdPdTbKk/yaFfaZPhVQQYaUAtvWigv3/9I1a7++0kQUfcZrIN/4g/RQygR2NqW2vAhHUCimJ9SFGn6VELpNrJZAe3hXkJ4ftR9YHvLxx2FYXVN3zxV5kX5BJl5ka1u3lF1md2++Efhyi6haEdGHDqNM2uhSPuyP8s52GerE4OnwHQLi8ZyGvD6sBiorYDMlhzkRaFvAiO0vh9lBy46mo2bSFIL1FiE3cEfimIsHYQ8mwQfcu/od7Gda4OCDl8IgUuFefiZQW8cORoqtUyDXJZMBzRN8NmMocnz1btQgGW8GXVQXBdLPutrmsKWLUC3oAKRTgc0uP16Qufx/7lY6Bv/wU52Kk6XScva9Jy4W0snp0xSxtT6d2T9sZhxw4paacCs1iZjvjtbxQKbmWf+OiHi1KIMZ79wfxRGCNy6/mNh6j620LHGUXHwWi/Ri3zWjPW4IuoG46aIX3Fnc9JyacaqaMJjTCkk6OqBjqwAQBnWkju/GDdFn/S6d3rL12+DoCxVHoZrqQwnrYtXHBT78uw9ULyUasT/UeoDbFXFKK9ENVumeGEPPoVv/+TPVsR5Gn7OojLorJqy0TJZAQWrcHXssYzSwzTMydxoNoTfY6MPQRFdLWbbJc0cs49PgpNLFi6wAGV69bvpxq/1utGypEtH0bf9oLNmjfDWltW8Vs3eF6a/2+7fRobfG4T0gCAn9xcWBryB4MDrrv7qK8e",
  "Expiration" : "2023-11-06T17:28:26Z"
}
" http.response.header.server=EC2ws http.response_content_length=1594 tf_rpc=ConfigureProvider http.response.header.date="Mon, 06 Nov 2023 11:30:56 GMT" http.response.header.last_modified="Mon, 06 Nov 2023 11:10:28 GMT" rpc.system=aws-api tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 aws.region= http.status_code=200 tf_aws.sdk=aws-sdk-go-v2 tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-06T11:30:56.042Z
2023-11-06T11:30:56.042Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: Retrieved credentials: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:39 @module=aws.aws-base tf_aws.credentials_source=EC2RoleProvider tf_provider_addr=registry.terraform.io/hashicorp/aws tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.042Z
2023-11-06T11:30:56.042Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Loading configuration: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp=2023-11-06T11:30:56.042Z
2023-11-06T11:30:56.044Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Creating AWS SDK v1 session: @module=aws tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @caller=github.com/hashicorp/terraform-provider-aws/internal/conns/config.go:149 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.044Z
2023-11-06T11:30:56.044Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Retrieving AWS account details: @module=aws tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/terraform-provider-aws/internal/conns/config.go:164 tf_rpc=ConfigureProvider tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 timestamp=2023-11-06T11:30:56.044Z
2023-11-06T11:30:56.044Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: Retrieving caller identity from STS: tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 tf_rpc=ConfigureProvider @module=aws.aws-base @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 timestamp=2023-11-06T11:30:56.044Z
2023-11-06T11:30:56.045Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: http.request.body="Action=GetCallerIdentity&Version=2011-06-15
" http.request.header.content_type=application/x-www-form-urlencoded http.request.header.x_amz_date=20231106T113056Z rpc.method=GetCallerIdentity rpc.service=STS http.method=POST @module=aws.aws-base http.request_content_length=43 rpc.system=aws-api tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 tf_aws.sdk=aws-sdk-go-v2 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.1 os/linux lang/go#1.20.10 md/GOOS#linux md/GOARCH#amd64 api/sts#1.25.0" http.request.header.amz_sdk_request="attempt=1; max=25" http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************GJMD/20231106/eu-central-1/sts/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" http.request.header.x_amz_security_token=***** http.url=https://sts.eu-central-1.amazonaws.com/ net.peer.name=sts.eu-central-1.amazonaws.com tf_rpc=ConfigureProvider aws.region=eu-central-1 http.request.header.amz_sdk_invocation_id=b3b5a699-d59b-4640-b476-12d4013e1fd6 timestamp=2023-11-06T11:30:56.045Z
2023-11-06T11:30:56.076Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:47 http.response.header.x_amzn_requestid=a241f4a4-3938-4e8f-bfb5-16fc869942be rpc.system=aws-api tf_provider_addr=registry.terraform.io/hashicorp/aws http.status_code=200 rpc.method=GetCallerIdentity http.duration=30 http.response.body="<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c</Arn>
    <UserId>AROA*************XBNG:i-01260e96be8c8250c</UserId>
    <Account>405223848550</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>a241f4a4-3938-4e8f-bfb5-16fc869942be</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>
" http.response.header.content_type=text/xml http.response_content_length=480 aws.region=eu-central-1 rpc.service=STS tf_aws.sdk=aws-sdk-go-v2 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 @module=aws.aws-base http.response.header.date="Mon, 06 Nov 2023 11:30:55 GMT" tf_rpc=ConfigureProvider timestamp=2023-11-06T11:30:56.075Z
2023-11-06T11:30:56.076Z [INFO]  provider.terraform-provider-aws_v5.24.0_x5: Retrieved caller identity from STS: @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/tf_logger.go:39 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=35cd8dab-8b96-b327-da8b-d5693849f4e3 timestamp=2023-11-06T11:30:56.076Z
2023-11-06T11:30:56.077Z [DEBUG] Resource instance state not found for node "data.aws_vpc.k8s_vpc", instance data.aws_vpc.k8s_vpc
2023-11-06T11:30:56.077Z [DEBUG] ReferenceTransformer: "data.aws_vpc.k8s_vpc" references: []
2023-11-06T11:30:56.077Z [DEBUG] Resource instance state not found for node "data.aws_caller_identity.target_account", instance data.aws_caller_identity.target_account
2023-11-06T11:30:56.077Z [DEBUG] ReferenceTransformer: "data.aws_caller_identity.target_account" references: []
2023-11-06T11:30:56.079Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: http.flavor=1.1 http.request_content_length=208 net.peer.name=ec2.eu-central-1.amazonaws.com rpc.system=aws-api tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ReadDataSource @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/[email protected]/logger.go:109 aws.region=eu-central-1 http.method=POST http.request.header.x_amz_date=20231106T113056Z http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.47.1 (go1.20.10; linux; amd64)" tf_req_id=32b875b2-da1d-9cf8-713f-3640d24967cb tf_data_source_type=aws_vpc @module=aws http.request.body="Action=DescribeVpcs&Filter.1.Name=tag%3AClassification&Filter.1.Value.1=PROD&Filter.2.Name=tag%3ALower_Role&Filter.2.Value.1=integration&Filter.3.Name=tag%3AApplication&Filter.3.Value.1=K8S&Version=2016-11-15
" http.url=https://ec2.eu-central-1.amazonaws.com/ rpc.service=EC2 tf_aws.sdk=aws-sdk-go tf_provider_addr=registry.terraform.io/hashicorp/aws http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************GJMD/20231106/eu-central-1/ec2/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" http.request.header.content_type="application/x-www-form-urlencoded; charset=utf-8" http.request.header.x_amz_security_token=***** rpc.method=DescribeVpcs timestamp=2023-11-06T11:30:56.079Z
2023-11-06T11:30:56.079Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Request Sent: @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/[email protected]/logger.go:109 aws.region=eu-central-1 http.request.body="Action=GetCallerIdentity&Version=2011-06-15
" http.request.header.content_type="application/x-www-form-urlencoded; charset=utf-8" http.request_content_length=43 tf_rpc=ReadDataSource http.flavor=1.1 http.request.header.x_amz_date=20231106T113056Z http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/5.24.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.47.1 (go1.20.10; linux; amd64)" http.method=POST http.url=https://sts.amazonaws.com/ rpc.system=aws-api tf_aws.signing_region=us-east-1 tf_req_id=0fbe919f-dcfa-e621-b9c2-a71375aa09b8 tf_mux_provider=*proto5server.Server rpc.service=STS tf_provider_addr=registry.terraform.io/hashicorp/aws @module=aws http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************GJMD/20231106/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" rpc.method=GetCallerIdentity http.request.header.x_amz_security_token=***** net.peer.name=sts.amazonaws.com tf_aws.sdk=aws-sdk-go tf_data_source_type=aws_caller_identity timestamp=2023-11-06T11:30:56.079Z

Then the section containing errors :

2023-11-06T11:30:56.111Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: rpc.system=aws-api tf_data_source_type=aws_vpc http.response.body="<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>UnauthorizedOperation</Code><Message>You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action</Message></Error></Errors><RequestID>274df92a-ee63-47cc-9d63-9b7a3f9b27f7</RequestID></Response>
" http.response.header.strict_transport_security="max-age=31536000; includeSubDomains" http.status_code=403 tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/[email protected]/logger.go:157 @module=aws http.response.header.date="Mon, 06 Nov 2023 11:30:55 GMT" http.response.header.server=AmazonEC2 http.response.header.vary=accept-encoding tf_mux_provider=*schema.GRPCProviderServer tf_req_id=32b875b2-da1d-9cf8-713f-3640d24967cb http.duration=31 http.response.header.cache_control="no-cache, no-store" http.response.header.content_type=text/xml;charset=UTF-8 tf_aws.sdk=aws-sdk-go tf_rpc=ReadDataSource aws.region=eu-central-1 http.response.header.x_amzn_requestid=274df92a-ee63-47cc-9d63-9b7a3f9b27f7 rpc.method=DescribeVpcs rpc.service=EC2 timestamp=2023-11-06T11:30:56.111Z
2023-11-06T11:30:56.111Z [ERROR] provider.terraform-provider-aws_v5.24.0_x5: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR tf_proto_version=5.4 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=32b875b2-da1d-9cf8-713f-3640d24967cb tf_rpc=ReadDataSource @module=sdk.proto diagnostic_summary="reading EC2 VPC: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action
	status code: 403, request id: 274df92a-ee63-47cc-9d63-9b7a3f9b27f7" tf_data_source_type=aws_vpc @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 timestamp=2023-11-06T11:30:56.111Z
2023-11-06T11:30:56.111Z [ERROR] vertex "data.aws_vpc.k8s_vpc" error: reading EC2 VPC: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action
	status code: 403, request id: 274df92a-ee63-47cc-9d63-9b7a3f9b27f7
2023-11-06T11:30:56.111Z [ERROR] vertex "data.aws_vpc.k8s_vpc" error: reading EC2 VPC: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action
	status code: 403, request id: 274df92a-ee63-47cc-9d63-9b7a3f9b27f7
2023-11-06T11:30:56.111Z [ERROR] vertex "data.aws_vpc.k8s_vpc (expand)" error: reading EC2 VPC: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action
	status code: 403, request id: 274df92a-ee63-47cc-9d63-9b7a3f9b27f7
2023-11-06T11:30:56.461Z [DEBUG] provider.terraform-provider-aws_v5.24.0_x5: HTTP Response Received: http.response_content_length=480 rpc.method=GetCallerIdentity rpc.service=STS tf_aws.signing_region=us-east-1 @module=aws http.response.body="<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c</Arn>
    <UserId>AROA*************XBNG:i-01260e96be8c8250c</UserId>
    <Account>405223848550</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>be695a72-fdb6-411b-ac72-a0eb3c270e18</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>
" http.response.header.date="Mon, 06 Nov 2023 11:30:55 GMT" tf_rpc=ReadDataSource @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/[email protected]/logger.go:157 rpc.system=aws-api tf_provider_addr=registry.terraform.io/hashicorp/aws http.status_code=200 tf_req_id=0fbe919f-dcfa-e621-b9c2-a71375aa09b8 http.duration=382 http.response.header.content_type=text/xml http.response.header.x_amzn_requestid=be695a72-fdb6-411b-ac72-a0eb3c270e18 tf_mux_provider=*proto5server.Server aws.region=eu-central-1 tf_aws.sdk=aws-sdk-go tf_data_source_type=aws_caller_identity timestamp=2023-11-06T11:30:56.461Z
2023-11-06T11:30:56.462Z [INFO]  backend/local: plan operation completed
╷
│ Error: reading EC2 VPC: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::405223848550:assumed-role/gtp-jenkins-slaves-instance-profile/i-01260e96be8c8250c is not authorized to perform: ec2:DescribeVpcs because no identity-based policy allows the ec2:DescribeVpcs action
│ 	status code: 403, request id: 274df92a-ee63-47cc-9d63-9b7a3f9b27f7
│ 
│   with data.aws_vpc.k8s_vpc,
│   on data.tf line 4, in data "aws_vpc" "k8s_vpc":
│    4: data "aws_vpc" "k8s_vpc" {
│ 
╵
2023-11-06T11:30:56.462Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-11-06T11:30:56.467Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.24.0/linux_amd64/terraform-provider-aws_v5.24.0_x5 pid=128
2023-11-06T11:30:56.467Z [DEBUG] provider: plugin exited

What is strange is that the error seems to indicate that the instance profile has not got the permission to do the action, whereas in reality, the instance profile should only be used to assume a role to do this specific action... it seems that the assume role is not taken into account...

With exactly the same EC2 and TF code :

If I use AWS CLI from the same environment to do a "DescribeVPCs" command : no error
If I pin terraform aws provider the 5.23.0 : no error

I tried setting the hop to 2 : same error

Farek · 2023-11-06T13:37:21Z

@yogeek I had a similar setup as You and was able to fix it.

I have a file that is pointing to AWS_SHARED_CREDENTIALS_FILE with credentials for my default user:

[common-profile]
aws_access_key_id=<KEY>
aws_secret_access_key=<SECRET>

And AWS_CONFIG_FILE pointing to my aws config:

[profile specific-profile]
    role_arn=<ROLE_ARN>
    source_profile=common-profile
    region=<REGION>

Running terraform with this setup was failing with:

│ Error: Invalid provider configuration
│
│ Provider "registry.terraform.io/hashicorp/aws" requires explicit
│ configuration. Add a provider block to the root module and configure the
│ provider's required arguments as described in the provider documentation.
│
╵
╷
│ Error: No valid credential sources found
│
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on <empty> line 0:
│   (source code not available)
│
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│
│ Error: failed to refresh cached credentials, no EC2 IMDS role found,
│ operation error ec2imds: GetMetadata, request canceled, context deadline
│ exceeded

What solved this problem in my case was to remove all whitespaces in profile definition:

[profile specific-profile]
role_arn=<ROLE_ARN>
source_profile=common-profile
region=<REGION>

I hope that it will help 😃

yogeek · 2023-11-06T15:03:36Z

@Farek oh yes indeed, it works by removing all the spaces (leading + trailing) 👏

I just read again a previous comment from @gdavison that pointed the same thing, but I did not read carefully at the time being 😅

So thank you to you both 👍

adamaziz15 · 2023-11-06T16:22:10Z

@gdavison

Looks like I was having the same issue as @yogeek, there was some whitespace in my creds file that I didn't notice because I'm writing it programmatically. Things are working now after removing that whitespace 🙏

gdavison · 2024-01-10T00:30:45Z

It looks like the fixes to the upstream AWS SDK have resolved this issue. If you're still experiencing this problem, please open a new issue

github-actions · 2024-02-09T02:01:11Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

adamaziz15 added the bug Addresses a defect in current functionality. label Nov 2, 2023

terraform-aws-provider bot added the needs-triage Waiting for first response or review from a maintainer. label Nov 3, 2023

ewbankkit added provider Pertains to the provider itself, rather than any interaction with AWS. authentication Pertains to authentication; to the provider itself of otherwise. labels Nov 3, 2023

ewbankkit added regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. and removed needs-triage Waiting for first response or review from a maintainer. labels Nov 3, 2023

terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Nov 3, 2023

gdavison self-assigned this Nov 3, 2023

ewbankkit mentioned this issue Nov 5, 2023

[Bug]: 5.24 provider stalls during plan phase #34257

Closed

gdavison added the upstream Addresses functionality related to the cloud provider. label Nov 6, 2023

gdavison mentioned this issue Nov 7, 2023

Rolls back AWS SDK modules with shared config parsing bug #34300

Merged

gdavison closed this as completed Jan 10, 2024

github-actions bot locked as resolved and limited conversation to collaborators Feb 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Authentication failure on v5.24.0 - incorrectly trying to use ec2 IMDS #34234

[Bug]: Authentication failure on v5.24.0 - incorrectly trying to use ec2 IMDS #34234

adamaziz15 commented Nov 2, 2023

github-actions bot commented Nov 3, 2023

ewbankkit commented Nov 3, 2023 •

edited

Loading

yogeek commented Nov 3, 2023 •

edited

Loading

gdavison commented Nov 3, 2023

yogeek commented Nov 3, 2023

gdavison commented Nov 3, 2023

gdavison commented Nov 3, 2023

adamaziz15 commented Nov 4, 2023 •

edited

Loading

adamaziz15 commented Nov 4, 2023

yogeek commented Nov 6, 2023 •

edited

Loading

Farek commented Nov 6, 2023

yogeek commented Nov 6, 2023 •

edited

Loading

adamaziz15 commented Nov 6, 2023

gdavison commented Jan 10, 2024

github-actions bot commented Feb 9, 2024

[Bug]: Authentication failure on v5.24.0 - incorrectly trying to use ec2 IMDS #34234

[Bug]: Authentication failure on v5.24.0 - incorrectly trying to use ec2 IMDS #34234

Comments

adamaziz15 commented Nov 2, 2023

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

github-actions bot commented Nov 3, 2023

Community Note

ewbankkit commented Nov 3, 2023 • edited Loading

yogeek commented Nov 3, 2023 • edited Loading

gdavison commented Nov 3, 2023

yogeek commented Nov 3, 2023

gdavison commented Nov 3, 2023

gdavison commented Nov 3, 2023

adamaziz15 commented Nov 4, 2023 • edited Loading

adamaziz15 commented Nov 4, 2023

yogeek commented Nov 6, 2023 • edited Loading

Farek commented Nov 6, 2023

yogeek commented Nov 6, 2023 • edited Loading

adamaziz15 commented Nov 6, 2023

gdavison commented Jan 10, 2024

github-actions bot commented Feb 9, 2024

ewbankkit commented Nov 3, 2023 •

edited

Loading

yogeek commented Nov 3, 2023 •

edited

Loading

adamaziz15 commented Nov 4, 2023 •

edited

Loading

yogeek commented Nov 6, 2023 •

edited

Loading

yogeek commented Nov 6, 2023 •

edited

Loading