validator-plugin-aws

The AWS validator plugin ensures that your AWS environment matches a user-configurable expected state.

Description

The AWS validator plugin reconciles AwsValidator custom resources to perform the following validations against your AWS environment:

1. Ensure that one or more EC2 AMI(s) exist in a particular region.
2. Compare the IAM permissions associated with an IAM user / group / role / policy against an expected permission set.
3. Compare the usage for a particular service quota against the active quota to avoid unexpectedly hitting quota limits.
   • Essentially Quota Monitor for AWS but cheaper, simpler, and open source.
4. Compare the tags associated with a subnet against an expected tag set.

Each AwsValidator CR is (re)-processed every two minutes to continuously ensure that your AWS environment matches the expected state.

See the samples directory for example AwsValidator configurations.

Authn & Authz

Authentication details for the AWS validator controller are provided within each AwsValidator custom resource. AWS authentication can be configured either implicitly or explicitly. All supported options are detailed below:

• Implicit (AwsValidator.auth.implicit == true)
  • IAM roles for Amazon EC2
  • IAM roles for Amazon EKS
  • IAM roles for Service Accounts (OIDC)
    • In this scenario, a valid Service Account must be specified during plugin installation.
• Explicit - Kubernetes Secret (AwsValidator.auth.implicit == false && AwsValidator.auth.secretName != "")
  • Environment variables
  • Environment variables + role assumption via AWS STS
• Explicit - Inline (AwsValidator.auth.implicit == false && AwsValidator.auth.credentials != {})
  • Environment variables
  • Environment variables + role assumption via AWS STS

Note

See values.yaml for additional configuration details for each authentication option.

Configuring AWS SDK

• The max attempts setting for retries is set to 25. Use auth.MaxAttempts to increase this further or set any other desired value. Valid values are those gte 0.

Minimal AWS managed IAM permission policies by validation type

For validation to succeed, certain AWS managed permission policies must be attached to the principal used and/or assumed by the AWS validator controller. The minimal required IAM policies, broken out by validation category, are as follows:

• IAM
  • User Validation

    {
    	"Version": "2012-10-17",
    	"Statement": [
    		{
    			"Sid": "VisualEditor0",
    			"Effect": "Allow",
    			"Action": [
    				"iam:ListAttachedUserPolicies",
    				"iam:GetContextKeysForPrincipalPolicy",
    				"iam:GetPolicy",
    				"iam:GetPolicyVersion",
    				"iam:GetUser",
    				"iam:SimulatePrincipalPolicy"
    			],
    			"Resource": "arn:aws:iam::<ACCOUNT_ID>:user/*"
    		}
    	]
    }

  • Role Validation

    {
    	"Version": "2012-10-17",
    	"Statement": [
    		{
    			"Sid": "VisualEditor0",
    			"Effect": "Allow",
    			"Action": [
    				"iam:ListAttachedRolePolicies",
    				"iam:GetContextKeysForPrincipalPolicy",
    				"iam:GetPolicy",
    				"iam:GetPolicyVersion",
    				"iam:GetRole",
    				"iam:SimulatePrincipalPolicy"
    			],
    			"Resource": "arn:aws:iam::<ACCOUNT_ID>:role/*"
    		}
    	]
    }

  • Group Validation

    {
    	"Version": "2012-10-17",
    	"Statement": [
    		{
    			"Sid": "VisualEditor0",
    			"Effect": "Allow",
    			"Action": [
    				"iam:ListAttachedGroupPolicies",
    				"iam:GetGroup",
    				"iam:GetPolicy",
    				"iam:GetPolicyVersion",
    				"iam:SimulatePrincipalPolicy"
    			],
    			"Resource": "arn:aws:iam::<ACCOUNT_ID>:group/*"
    		}
    	]
    }

• Service Quotas
  • Requires the following IAM policies:
    • AmazonEC2ReadOnlyAccess
    • AmazonElasticFileSystemReadOnlyAccess
    • ElasticLoadBalancingReadOnly
    • ServiceQuotasReadOnlyAccess
  • Combined JSON policy:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "AmazonEC2ReadOnlyAccess1",
                "Effect": "Allow",
                "Action": "ec2:Describe*",
                "Resource": "*"
            },
            {
                "Sid": "AmazonEC2ReadOnlyAccess2",
                "Effect": "Allow",
                "Action": [
                    "elasticloadbalancing:Describe*",
                    "elasticloadbalancing:Get*"
                ],
                "Resource": "*"
            },
            {
                "Sid": "AmazonEC2ReadOnlyAccess3",
                "Effect": "Allow",
                "Action": [
                    "cloudwatch:ListMetrics",
                    "cloudwatch:GetMetricData",
                    "cloudwatch:GetMetricStatistics",
                    "cloudwatch:Describe*",
                    "cloudwatch:DescribeAlarmsForMetric"
                ],
                "Resource": "*"
            },
            {
                "Sid": "AmazonEC2ReadOnlyAccess4",
                "Effect": "Allow",
                "Action": "autoscaling:Describe*",
                "Resource": "*"
            },
            {
                "Sid": "AmazonElasticFileSystemReadOnlyAccess",
                "Effect": "Allow",
                "Action": [
                    "elasticfilesystem:DescribeAccountPreferences",
                    "elasticfilesystem:DescribeBackupPolicy",
                    "elasticfilesystem:DescribeFileSystems",
                    "elasticfilesystem:DescribeFileSystemPolicy",
                    "elasticfilesystem:DescribeLifecycleConfiguration",
                    "elasticfilesystem:DescribeMountTargets",
                    "elasticfilesystem:DescribeMountTargetSecurityGroups",
                    "elasticfilesystem:DescribeTags",
                    "elasticfilesystem:DescribeAccessPoints",
                    "elasticfilesystem:DescribeReplicationConfigurations",
                    "elasticfilesystem:ListTagsForResource",
                    "kms:ListAliases"
                ],
                "Resource": "*"
            },
            {
                "Sid": "ServiceQuotasReadOnlyAccess",
                "Effect": "Allow",
                "Action": [
                    "cloudformation:DescribeAccountLimits",
                    "dynamodb:DescribeLimits",
                    "iam:GetAccountSummary",
                    "kinesis:DescribeLimits",
                    "organizations:DescribeAccount",
                    "organizations:DescribeOrganization",
                    "organizations:ListAWSServiceAccessForOrganization",
                    "rds:DescribeAccountAttributes",
                    "route53:GetAccountLimit",
                    "tag:GetTagKeys",
                    "tag:GetTagValues",
                    "servicequotas:GetAssociationForServiceQuotaTemplate",
                    "servicequotas:GetAWSDefaultServiceQuota",
                    "servicequotas:GetRequestedServiceQuotaChange",
                    "servicequotas:GetServiceQuota",
                    "servicequotas:GetServiceQuotaIncreaseRequestFromTemplate",
                    "servicequotas:ListAWSDefaultServiceQuotas",
                    "servicequotas:ListRequestedServiceQuotaChangeHistory",
                    "servicequotas:ListRequestedServiceQuotaChangeHistoryByQuota",
                    "servicequotas:ListServices",
                    "servicequotas:ListServiceQuotas",
                    "servicequotas:ListServiceQuotaIncreaseRequestsInTemplate",
                    "servicequotas:ListTagsForResource"
                ],
                "Resource": "*"
            }
        ]
    }

• Tags
  • Requires the AmazonVPCReadOnlyAccess IAM policy, which is a subset of AmazonEC2ReadOnlyAccess

Note

Validation can be successful with custom IAM policies that are even more restrictive than the AWS managed policies listed above, but these will vary on a case-by-case basis and hence are undocumented for the sake of maintainability.

Supported Service Quotas by AWS Service

EC2:

• EC2-VPC Elastic IPs
• Public AMIs

EFS:

• File systems per account

ELB:

• Application Load Balancers per Region
• Classic Load Balancers per Region
• Network Load Balancers per Region

VPC:

• Internet gateways per Region
• Network interfaces per Region
• VPCs per Region
• NAT gateways per Availability Zone
• Subnets per VPC

Installation

The AWS validator plugin is meant to be installed by validator (via a ValidatorConfig), but it can also be installed directly as follows:

helm repo add validator-plugin-aws https://validator-labs.github.io/validator-plugin-aws
helm repo update
helm install validator-plugin-aws validator-plugin-aws/validator-plugin-aws -n validator-plugin-aws --create-namespace

Development

You’ll need a Kubernetes cluster to run against. You can use kind to get a local cluster for testing, or run against a remote cluster. Note: Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster kubectl cluster-info shows).

Running on the cluster

1. Install Instances of Custom Resources:

kubectl apply -f config/samples/

2. Build and push your image to the location specified by IMG:

make docker-build docker-push IMG=<some-registry>/validator-plugin-aws:tag

3. Deploy the controller to the cluster with the image specified by IMG:

make deploy IMG=<some-registry>/validator-plugin-aws:tag

Uninstall CRDs

To delete the CRDs from the cluster:

make uninstall

Undeploy controller

UnDeploy the controller from the cluster:

make undeploy

How it works

This project aims to follow the Kubernetes Operator pattern.

It uses Controllers, which provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.

Test It Out

1. Install the CRDs into the cluster:

make install

2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):

make run

NOTE: You can also run this in one step by running: make install run

Modifying the API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

make manifests

NOTE: Run make --help for more information on all potential make targets

More information can be found via the Kubebuilder Documentation

Contributing

All contributions are welcome! Feel free to reach out on the Spectro Cloud community Slack.

Make sure pre-commit is installed.

Install the pre-commit scripts:

pre-commit install --hook-type commit-msg
pre-commit install --hook-type pre-commit

License

Copyright 2023.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.