Support using custom KMS keys to build private stemcells #31
+23,830
−8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
as we paused the aws light stemcell builder tests. in the publish pipeline. |
mvach
force-pushed
the
support-custom-kms-keys
branch
2 times, most recently
from
5a6af98 to
4d66c9b
Compare
jpalermo
requested review from
a team,
selzoc and
lnguyen
and removed request for
a team
lnguyen
requested changes
Nov 14, 2023
selzoc
requested changes
Nov 14, 2023
When building a private stemcells, the builder currently uses a managed KMS key which is default AWS account key. Using this key prevents sharing stemcells across accounts. Therefore we add the custom KMS key support.
mvach
force-pushed
the
support-custom-kms-keys
branch
from
4d66c9b to
fed3ae5
Compare
|
We're seeing failures with the drivers test. This may well be an issue with our setup, I'm still investigating. Posting the info here for posterity: Test script#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$( cd "$(dirname "${0}")" && pwd )"
source "${SCRIPT_DIR}/utils.sh"
tmp_dir="$(mktemp -d /tmp/stemcell_builder.XXXXXXX)"
trap '{ rm -rf ${tmp_dir}; }' EXIT
# US Regions
# shellcheck disable=SC2154
export AWS_ACCESS_KEY_ID="${access_key}"
# shellcheck disable=SC2154
export AWS_SECRET_ACCESS_KEY="${secret_key}"
# shellcheck disable=SC2154
export AWS_BUCKET_NAME="${bucket_name}"
# shellcheck disable=SC2154
export AWS_REGION="${region}"
export AWS_ROLE_ARN="${role_arn:-}"
# shellcheck disable=SC2154
export AWS_DESTINATION_REGION="${copy_region}"
# shellcheck disable=SC2154
export AWS_KMS_KEY_ID="${kms_key_id}"
# Fixtures
# shellcheck disable=SC2154
export S3_MACHINE_IMAGE_URL="${uploaded_machine_image_url}"
export S3_MACHINE_IMAGE_FORMAT="${uploaded_machine_image_format:=RAW}"
# shellcheck disable=SC2154
export EBS_VOLUME_ID="${existing_volume_id}"
# shellcheck disable=SC2154
export EBS_SNAPSHOT_ID="${existing_snapshot_id}"
# shellcheck disable=SC2154
export AMI_FIXTURE_ID="${ami_fixture_id}"
echo "Downloading machine image"
export MACHINE_IMAGE_PATH="${tmp_dir}/image.iso"
export MACHINE_IMAGE_FORMAT="RAW"
wget http://tinycorelinux.net/7.x/x86_64/archive/7.1/TinyCorePure64-7.1.iso \
-O "${MACHINE_IMAGE_PATH}"
echo "Running driver tests"
(
cd builder-src
# Run all driver specs in parallel to reduce test time
spec_count="$(grep "It(" -r driver | wc -l)"
go run github.com/onsi/ginkgo/v2/ginkgo -nodes "${spec_count}" -r driver
)Test output |
|
Quoting from email:
Unfortunately both hard-coding a known aws account id, as well as fetching one via awsAccount = os.Getenv("AWS_ACCOUNT")
if awsAccount == "" {
GinkgoWriter.Printf("AWS_ACCOUNT was empty, attempting sts.GetCallerIdentity()")
session, err := session.NewSession()
Expect(err).NotTo(HaveOccurred())
result, err := sts.New(session).GetCallerIdentity(&sts.GetCallerIdentityInput{})
Expect(err).NotTo(HaveOccurred())
awsAccount = *result.Account
GinkgoWriter.Printf("GetCallerIdentity() return %s", awsAccount)
}
Expect(awsAccount).ToNot(BeEmpty(), "AWS_ACCOUNT must be set")Results in errors: Perhaps there are additional constraints on the |
|
We fixed a couple of the test failures here by limiting which tests are using the sharing functionality. But the two tests that are testing sharing and kms keys still get this: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When building a private stemcell, the aws light stemcell builder currently uses a managed KMS key which is default AWS account key. Using this key prevents sharing stemcells across accounts. Therefore we add the custom KMS key support.
This PR belongs to Issue #30 and it relevant for building FPIS light stemcells.