Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarity on aws_role_arn #277

Closed
luis-n4489 opened this issue May 5, 2021 · 8 comments · Fixed by #323
Closed

Clarity on aws_role_arn #277

luis-n4489 opened this issue May 5, 2021 · 8 comments · Fixed by #323

Comments

@luis-n4489
Copy link

luis-n4489 commented May 5, 2021

I am assuming that the aws_role_arn is NOT assuming the role (bc the ec2 instance has access to assume it) and should be able to go and fetch the docker image from ecr without having to add the access key and secret.

I am trying to just use
resources:

name: myrepo
type: registry-image
source:
repository: '12345678.dkr.ecr.us-west-1.amazonaws.com/mytestrepo'
aws_role_arn: 'arn:aws:iam::12345678:role/testrole_has_access'
aws_region: 'us-west-1'

I can successfully do any pull or push via aws cli and assuming this role within the workers, but when this resources tries i get failed.. ERRo[0000] checking origin 12345678.dkr.ecr.us-west-1.amazonaws.com/mytestrepo failed: list repository tags: GET https://12345678.dkr.ecr.us-west-1.amazonaws.com/mytestrepo/tags/list?n=1000L unexpected status code 401 Unauthorized: Not authorized

fyi, i gave all perms to the role - even getauthtoken. all this works via cli but not this resource. (If thats what aws_role_arn is used for)

Concourse 7.2.0

@luis-n4489
Copy link
Author

Read a lot regarding security and etc in past open issues, but no final thoughts on where it's going. Are there any updates regarding the use of aws_role_arn for assuming the role?

@jorymorrison
Copy link

+1

@luis-n4489
Copy link
Author

luis-n4489 commented Oct 24, 2021

I think the aws_role_arn still does not work =(
ERRO[0000] pushing image failed: pushing tag(s): HEAD https://1234567890.dkr.ecr.us-east-1.amazonaws.com/v2/test/blobs/sha256:1234: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)

@taylorsilva
Copy link
Member

Leaving a message here to let folks know that the core team doesn't use AWS so we're quite out of our depth here. Help from someone in the community who can troubleshoot this against AWS would be great. We'd be happy to help from the code side, giving pointers and showing how to run thing if needed.

@samed
Copy link

samed commented Nov 3, 2021

Leaving a message here to let folks know that the core team doesn't use AWS so we're quite out of our depth here. Help from someone in the community who can troubleshoot this against AWS would be great. We'd be happy to help from the code side, giving pointers and showing how to run thing if needed.

@taylorsilva I've managed to get it working by just relying on req.Source.AwsRegion in the source, so in this way, I made it fallback to instance IAM profile without using hardcoded credentials. AWS-SDK has the following precedence (https://docs.aws.amazon.com/sdk-for-go/api/aws/session/) for credentials/sessions:

  • Environment Variables
  • Shared Credentials file
  • Shared Configuration file (if SharedConfig is enabled)
  • EC2 Instance Metadata (credentials only)

Here's my patch, which is just an (ugly) workaround.

diff --git i/commands/check.go w/commands/check.go
index fa93368..72512d1 100644
--- i/commands/check.go
+++ w/commands/check.go
@@ -49,7 +49,7 @@ func (c *Check) Execute() error {
                return fmt.Errorf("invalid payload: %s", err)
        }

-       if req.Source.AwsAccessKeyId != "" && req.Source.AwsSecretAccessKey != "" && req.Source.AwsRegion != "" {
+       if req.Source.AwsRegion != "" {
                if !req.Source.AuthenticateToECR() {
                        return fmt.Errorf("cannot authenticate with ECR")
                }
diff --git i/commands/in.go w/commands/in.go
index 3fc63d7..94d19d4 100644
--- i/commands/in.go
+++ w/commands/in.go
@@ -65,7 +65,7 @@ func (i *In) Execute() error {

        dest := i.args[1]

-       if req.Source.AwsAccessKeyId != "" && req.Source.AwsSecretAccessKey != "" && req.Source.AwsRegion != "" {
+       if req.Source.AwsRegion != "" {
                if !req.Source.AuthenticateToECR() {
                        return fmt.Errorf("cannot authenticate with ECR")
                }
diff --git i/commands/out.go w/commands/out.go
index 1f87ea3..3bad96c 100644
--- i/commands/out.go
+++ w/commands/out.go
@@ -63,7 +63,7 @@ func (o *Out) Execute() error {

        src := o.args[1]

-       if req.Source.AwsAccessKeyId != "" && req.Source.AwsSecretAccessKey != "" && req.Source.AwsRegion != "" {
+       if req.Source.AwsRegion != "" {
                if !req.Source.AuthenticateToECR() {
                        return fmt.Errorf("cannot authenticate with ECR")
                }
diff --git i/types.go w/types.go
index c8f6f65..6a88134 100644
--- i/types.go
+++ w/types.go
@@ -14,7 +14,6 @@ import (
        "strings"

        "github.com/aws/aws-sdk-go/aws"
-       "github.com/aws/aws-sdk-go/aws/credentials"
        "github.com/aws/aws-sdk-go/aws/credentials/stscreds"
        "github.com/aws/aws-sdk-go/aws/session"
        "github.com/aws/aws-sdk-go/service/ecr"
@@ -301,7 +300,6 @@ func (source *Source) AuthenticateToECR() bool {

        mySession := session.Must(session.NewSession(&aws.Config{
                Region:      aws.String(source.AwsRegion),
-               Credentials: credentials.NewStaticCredentials(source.AwsAccessKeyId, source.AwsSecretAccessKey, source.AwsSessionToken),
        }))

        // Note: This implementation gives precedence to `aws_role_arn` since it

@luis-n4489
Copy link
Author

I can confirm above fix works!

@jprelph
Copy link

jprelph commented Dec 16, 2021

I couldn't think of a nice way of doing this neatly without a specific flag so I've added a aws_use_instance_role option - which when set to true will skip the other cred options (just needs the region as well). I've raised a PR - #309

@akinggds
Copy link

I've been struggling to get that fix working. Can someone clarify how I should be using this resource?
I have the following:
resources.yml

  - name: reponame
    type: registry-image
    source:
      repository: "((aws_account_var)).dkr.ecr.eu-west-1.amazonaws.com/reponame"
      aws_region: "eu-west-1"
      aws_role_arn: "arn:aws:iam::((aws_account_var)):role/rolename"

resource_types.yml

  - name: reponame
    type: registry-image
    source:
      aws_role_arn: 'arn:aws:iam::((aws_account_var)):role/rolename'
      aws_region: eu-west-1
      repository: jprelph/registry-image-resource

When I run the pipeline it just hangs on preparing to build.

taylorsilva added a commit that referenced this issue Jul 19, 2022
Closes #277

Relaxes the requirement around having programmatic credentials. Only the
region is required now which will kick off allowing the aws sdk to try
it's various authentication methods. This allows the sdk to authenticate
with instance profiles on ec2 instances.

Signed-off-by: Taylor Silva <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants