backend/s3: Modernization tracking issue

[gdavison]

Terraform Version

N/A

Use Cases

The AWS Provider team is undertaking a project to modernize the S3 remote state backend. This will include:

• Using the AWS SDK for Go v2 instead of v1
• Renaming environment variables specific to the backend to not start with AWS_, as that namespace is reserved for AWS
• Updating configuration to mirror the AWS Provider

This issue will be used to track the overall project.

Attempted Solutions

N/A

Proposal

AWS SDK v2

• backend/s3: Upgrade to aws-sdk-go-base/v2 #30443
• backend/s3: Support assume_role_with_web_identity #31244

Expanded Auth Configuration

• backend/s3: Move assume role parameters to block #30495
• backend/s3: Deprecate AWS_???_ENDPOINT environment variables and replace with TF_S3_???_ENDPOINT #30479
• backend/s3: Add multiple credentials and config files parameters #30493
• backend/s3: Add support for overriding EC2 Metadata Service endpoint #30444
• backend/s3: Add parameter custom_ca_bundle #33689
• backend/s3: Support overriding EC2 Metadata endpoint #33690
• backend/s3: Add parameter sts_region #33693

Expanded Non-Auth Configuration

• backend/s3: Add HTTP parameters for parity with AWS Provider #30496
• backend/s3: Deprecate AWS_???_ENDPOINT environment variables and replace with TF_S3_???_ENDPOINT #30479

Consistency with AWS Provider

• backend/s3: Endpoint override parameter changes #30492
• backend/s3: S3 use path style configuration #30491
• backend/s3: Add allowed_account_ids and forbidden_account_ids parameters #33688

Enhancements

• backends/s3: Update fine-grained access control documentation with more complete examples #30445
• Address Backend S3: specify region in AWS CLI configuration instead of Terraform configuration #29254" Backend S3: specify region in AWS CLI configuration instead of Terraform configuration #29254 (comment)
• backend/s3: Allow specifying S3 Object using ARN #33686 Deferred
• backend/s3: Supports retry mode #33692

Bugs

• Confusing error message from the S3 backend with empty-string argument values #29557
• S3 backend - better error messages when using AWS credentials with no access to the configured S3 bucket #29008

References

No response

[lorengordon]

I don't suppose you'd consider adding credential_process as a configuration option as well? Similar to hashicorp/terraform-provider-aws#24885

[gdavison]

That's a good catch, @lorengordon. In general, we want to support the same authentication configuration methods across the S3 backend and the AWS provider.

I've created #33838

[gdavison]

Tasks completed

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.