Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove resources that are not needed #27

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion crossplane-api/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
IMAGE_NAME := dataservices
ECR_REPO := public.ecr.aws/w5n9a2g2/anynines
ECR_REPO := public.ecr.aws/w5n9a2g2/klutch

providerconfig:
# Populate Secrets and create PostgreSQL ProvideConfig using the instance name
Expand Down
97 changes: 14 additions & 83 deletions crossplane-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,9 @@ aws ecr-public get-login-password --region us-east-1 --profile=ECR | docker logi
### Push images to ECR

There are two ECR repositories, one is used to store provider images
(`public.ecr.aws/w5n9a2g2/anynines/provider-anynines`) and the other
(`public.ecr.aws/w5n9a2g2/klutch/provider-anynines`) and the other
one is used for the provider controller images
(`public.ecr.aws/w5n9a2g2/anynines/provider-anynines-controller`).
(`public.ecr.aws/w5n9a2g2/klutch/provider-anynines-controller`).


> **Important Note!**
Expand Down Expand Up @@ -202,7 +202,12 @@ In a separate terminal start an ssh tunnel to Service Broker
```bash
export SERVICE_INSTANCE_NAME=<dataservice name> #e.g. postgresql-ms-1686299661

export SERVICEBROKER_IP=$(ssh aws-s1-inception ". /var/vcap/store/jumpbox/home/a9s/bosh/envs/dsf2;bosh -d $SERVICE_INSTANCE_NAME instances | grep broker/" | awk '{print $4}')
```

**NOTE:** You would have to load the credentials to execute the bosh cli cmd in the BOSH director env.

```bash
export SERVICEBROKER_IP=$(ssh {IP of the virtual machine from where you can access the BOSH director};bosh -d $SERVICE_INSTANCE_NAME instances | grep broker/" | awk '{print $4}')

echo $SERVICEBROKER_IP

Expand All @@ -215,8 +220,12 @@ In a separate terminal start an ssh tunnel to Backup Manager

```bash
export SERVICE_INSTANCE_NAME=<dataservice name> #e.g. postgresql-ms-1686299661
```

**NOTE:** You would have to load the credentials to execute the bosh cli cmd in the BOSH director env.

export BACKUP_MANAGER_IP=$(ssh aws-s1-inception ". /var/vcap/store/jumpbox/home/a9s/bosh/envs/dsf2;bosh -d $SERVICE_INSTANCE_NAME instances | grep backup-manager/" | awk '{print $4}')
```bash
export BACKUP_MANAGER_IP=$(ssh {IP of the virtual machine from where you can access the BOSH director};bosh -d $SERVICE_INSTANCE_NAME instances | grep backup-manager/" | awk '{print $4}')

echo $BACKUP_MANAGER_IP

Expand Down Expand Up @@ -277,7 +286,7 @@ To install the configuration package (containing definitions and compositions),
1. Install the package via crossplane:

```bash
crossplane xpkg install configuration public.ecr.aws/w5n9a2g2/anynines/dataservices:v1.3.0
crossplane xpkg install configuration public.ecr.aws/w5n9a2g2/klutch/dataservices:v1.3.0
```

2. Install files directly:
Expand Down Expand Up @@ -487,84 +496,6 @@ you can restore a PostgreSQL Backup with the following command:
kubectl apply -f ./crossplane-api/examples/a9s/postgresql/restore-claim.yaml
```

## Usage - aws s3 provider

The aws s3 provider does require the helm and crossplane pre-requisites in order to be deployed

### AWS Access

Some of the pipelines require access to AWS EKS in order to be able to create clusters for testing
purposes. For these pipelines please apply a manifest using the following template:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: aws-secret
namespace: crossplane-system
type: Opaque
stringData:
creds: |-
[default]
AWS_ACCESS_KEY_ID: #<AWS_ACCESS_KEY_ID>
AWS_SECRET_ACCESS_KEY: #<AWS_SECRET_ACCESS_KEY>
config: |-
[default]
AWS_DEFAULT_REGION: eu-central-1
```

### Provision the aws s3 provider

The provider itself can be run by applying the provider file

```bash
kubectl apply -f ./crossplane-api/deploy/provider-aws.yaml
```

Verify the providers were created successfully. There should be 2 providers: provider-aws-s3 and upboard-provider-family-aws
```bash
kubectl get providers
```

Apply the provider config
```bash
kubectl apply -f ./crossplane-api/deploy/provider-aws-config.yaml
```

### Managing an s3 bucket

To create an s3 bucket, use the example object store yaml. This generates a hash for the name

```bash
kubectl create -f ./crossplane-api/examples/a8s/objectstore-claim.yaml
```

Check that the bucket was created successfully
```bash
kubectl get buckets
```

To delete the s3 bucket:

```bash
kubectl delete bucket <bucketname>
```

### Creating an s3 bucket policy

To add an s3 bucket policy as a service binding, use the example s3 service binding claim.
Make sure to update the claim with the bucket name and the iam role arn.

```bash
kubectl apply -f ./crossplane-api/examples/a8s/s3-servicebinding-claim.yaml
```

To remove the service binding

```bash
kubectl delete servicebinding/example-a8s-s3
```

## Update or Add a Service or Plan in a8s

In case of a Service or Plan is changed or a new one is added, it is essential
Expand Down
22 changes: 0 additions & 22 deletions crossplane-api/api/a8s/s3/composition.yaml

This file was deleted.

41 changes: 0 additions & 41 deletions crossplane-api/api/a8s/servicebinding/s3-composition.yaml

This file was deleted.

Binary file not shown.
2 changes: 1 addition & 1 deletion crossplane-api/api/common/servicebinding_definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ spec:
- region: &region ["eu-central-1", "us-east-1"]
- types: &serviceInstanceTypes ["postgresql", "mongodb",
"search", "logme2", "prometheus", "messaging", "mariadb",
"messaging", "s3"]
"messaging"]
properties:
spec:
x-kubernetes-validations:
Expand Down
17 changes: 0 additions & 17 deletions crossplane-api/api/iam/policy_composition.yaml

This file was deleted.

56 changes: 0 additions & 56 deletions crossplane-api/api/iam/policy_definition.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion crossplane-api/deploy/config-pkg-anynines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ kind: Configuration
metadata:
name: anynines-dataservices
spec:
package: public.ecr.aws/w5n9a2g2/anynines/dataservices:v1.3.0
package: public.ecr.aws/w5n9a2g2/klutch/dataservices:v1.3.0
2 changes: 1 addition & 1 deletion crossplane-api/deploy/provider-anynines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ kind: Provider
metadata:
name: provider-anynines
spec:
package: "public.ecr.aws/w5n9a2g2/anynines/provider-anynines:v1.3.0"
package: "public.ecr.aws/w5n9a2g2/klutch/provider-anynines:v1.3.0"
runtimeConfigRef:
name: provider-anynines
---
Expand Down
23 changes: 0 additions & 23 deletions crossplane-api/deploy/provider-aws-config.yaml

This file was deleted.

13 changes: 0 additions & 13 deletions crossplane-api/deploy/provider-aws.yaml

This file was deleted.

8 changes: 0 additions & 8 deletions crossplane-api/examples/a8s/objectstore-claim.yaml

This file was deleted.

13 changes: 0 additions & 13 deletions crossplane-api/examples/a8s/policy-claim.yaml

This file was deleted.

28 changes: 0 additions & 28 deletions crossplane-api/examples/a8s/s3-servicebinding-claim.yaml

This file was deleted.

Loading