Skip to content

Commit

Permalink
Fixed S3 buckets ARN - Lakeformation integration (#75)
Browse files Browse the repository at this point in the history
* added bucket policies to enforce in transit encryption for s3 buckets

* fixed cfn code style

* Delete to avoid conflicts

* Ingestion stage using sqoop

* removed kite jar and added installation instructions

* Enhanced EMR security configuration

* Fixing Merge conflicts

* Enhanced EMR security configuration

* Alternative dataset dependency

* Sqoop: Minor fixes and improvements

* Added athena get workgroup permissions

* deleted test transformation

* Removed support for python 3.6

* Fixed LF bucket registration ARN

When deploying the SDLF the s3 nested stack throws an error on all the    *LakeFormationS3Registration resources. 
 S3 bucket ARN changed form  arn:aws:::s3: to  arn:aws:s3:::, this change fixes the error

* New datasets not being deployed

New dataset creation skipped if previous datasets already exists
changed exit 0 to continue in order to deploy the next dataset if the current one already exists.

* Update deploy.sh

Co-authored-by: Arenas Garcia <[email protected]>
  • Loading branch information
emarenas and Arenas Garcia authored Nov 9, 2022
1 parent 991389f commit f903bc8
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ do
# Don't fail for no-op update
if [[ $update_output == *"ValidationError"* && $update_output == *"No updates"* ]] ; then
echo -e "\nFinished create/update - no updates to be performed";
exit 0;
continue
else
exit $status
fi
Expand Down Expand Up @@ -165,4 +165,4 @@ do
--condition-expression "attribute_exists(#N)"
echo "$TEAM_NAME-$DATASET_NAME DynamoDB Dataset entry updated"
fi
done
done

0 comments on commit f903bc8

Please sign in to comment.