diff --git a/config/install/README.md b/config/install/README.md index 343431719..79af821e4 100644 --- a/config/install/README.md +++ b/config/install/README.md @@ -1,101 +1,326 @@ -# Install Kuadrant and Sail via OLM +# Install and Configure Kuadrant and Sail via OLM and the CLI + +This document will walk you through setting up the required configuration to install kaudrant using kustomize or a tool that leverages kustomize such as kubectl along with OLM. It will walk you step by step through installation and building up your needed configuration. The full example is available to view and use here [Full AWS Example](https://github.com/Kuadrant/kuadrant-operator/tree/main/config/install/full-example-aws) ## Prerequisites -- Clone the[ Kuadrant-operator](https://github.com/Kuadrant/kuadrant-operator) repo -- OLM (operator lifecycle manager) -- cert-manager +- OCP or K8s cluster and CLI available. +- OLM is installed [operator lifecycle manager releases](https://github.com/operator-framework/operator-lifecycle-manager/releases) +- Gateway Provider Installed: By default this guide will install the [Sail Operator](https://github.com/istio-ecosystem/sail-operator) that will configure and install an Istio installation. Kuadrant is intended to work with [Istio](https://istio.io) or [Envoy Gateway](https://gateway.envoyproxy.io/) as a gateway provider before you can make use of Kuadrant one of these providers should be installed. +- (Optional) cert-manager: - [cert-manager Operator for Red Hat OpenShift](https://docs.openshift.com/container-platform/4.16/security/cert_manager_operator/cert-manager-operator-install.html) - [installing cert-manager via OperatorHub](https://cert-manager.io/docs/installation/operator-lifecycle-manager/) -- AWS, Azure or GCP with DNS capabilities. (Optional) -- Accessible Redis instance, for persistent storage for your rate limit counters. (Optional) +- (Optional) Access to AWS, Azure or GCP with DNS services. +- (Optional) Accessible Redis instance, for persistent storage for your rate limit counters. -> Note: By default the following guide will install the "latest" or "main" version of Kuadrant. To pick a specific version, change the image in the `config/install/standard/kuadrant-version.yaml`. All versions available can be found on the Kuadrant operator [release page](https://github.com/Kuadrant/kuadrant-operator/releases) > Note: for multiple clusters, it would make sense to do the installation via a tool like [argocd](https://argo-cd.readthedocs.io/en/stable/). For other methods of addressing multiple clusters take a look at the [kubectl docs](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) -## Setup the environment +## Basic Installation -> Note this uses the community operatorhub catalog. If you are installing on OpenShift and want to use the redhat provided catalog use the second command. +This first step will install just Kuadrant at a given released version (post v1.x) in the `kuadrant-system` namespace. There will be no credentials/dns providers configured (This is the most basic setup but means TLSPolicy and DNSPolicy will not be able to be used). This basic install will also setup a gateway provider via Istio and the Sail Operator. +Create the following `kustomization.yaml` in a directory locally. For the purpose of this doc, we will use: `kuadrant/install` (but if can be anything you would prefer). -```bash -# community catalog -kubectl apply -k config/install/standard -``` +> Setting the version to install: You can set the version of kuadrant to install by adding / changing the `?ref=v1.0.1`. + +```yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/standard?ref=v1.0.1 #set the versio by adding ?ref=v1.0.1 change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + #- https://github.com/Kuadrant/kuadrant-operator//config/install/openshift?ref=v1.0.1 #use if targeting an OCP cluster. Change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases). + +patches: # remove the subscription patch if you are installing a development version. It will then use the "preview" channel + - patch: |- + apiVersion: operators.coreos.com/v1alpha1 + kind: Subscription + metadata: + name: kuadrant + spec: + source: kuadrant-operator-catalog + sourceNamespace: kuadrant-system + name: kuadrant-operator + channel: 'stable' #set to preview if not using a release (for example if using main) + +``` + +And execute the following to apply it to a cluster: ```bash -# redhat catalog -kubectl apply -k config/install/openshift +# change the location depending on where you created the kustomization.yaml +kubectl apply -k + ``` -Verify both Kuadrant and sail operators are installed. Note, that this can take a while. You can also take a look at the subscription and installplan resource to help with debugging but the end state should be as below: +#### Verify the operators are installed: + +Once OLM has finished installing the operators (this can take several minutes). You should see the following in the kuadrant-system namespace: ```bash kubectl get deployments -n kuadrant-system - +## Output # NAME READY UP-TO-DATE AVAILABLE AGE # authorino-operator 1/1 1 1 83m # dns-operator-controller-manager 1/1 1 1 83m # kuadrant-console-plugin 1/1 1 1 83m # kuadrant-operator-controller-manager 1/1 1 1 83m # limitador-operator-controller-manager 1/1 1 1 83m + ``` +You can also view the subscription for information: +```bash +kubectl get subscription -n kuadrant-system -o=yaml + +``` + +### Install the operand components + +Kuadrant has 2 additional operand components that it manages (Authorino that provides data plane auth and Limitador that provides data plane rate limiting). To set these up lets add a new `kustomization.yaml` in a new sub directory. We will re-use this later for further configuration. We do this as a separate step as we want to have the operators installed and in place first. + +Add the following to your local directory. For the purpose of this doc, we will use: `kuadrant/configure/`. + +```yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/configure/standard?ref=v1.0.1 #change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + +``` + +Lets apply this to your cluster: ```bash -kubectl get deployments -n gateway-system +kubectl apply -k -# NAME READY UP-TO-DATE AVAILABLE AGE -# istiod 1/1 1 1 61s -# sail-operator 1/1 1 1 81m ``` -## Configure the installation +### Verify kuadrant is installed and ready: + +```bash +kubectl get kuadrant kuadrant -n kuadrant-system -o=wide + +# NAME STATUS AGE +# kuadrant Ready 109s -### TLS and DNS integration +``` +You should see the condition with type `Ready` with a message of `kuadrant is ready`. -Create the `$CLOUD_PROVIDER-credentials.env file` in the cloud provider directory `config/install/configure/$CLOUD_PROVIDER.` e.g. `aws-credentials.env` in the `config/install/configure/aws` directory. Apply the configuration for the desired cloud provider. Example AWS + +### Verify Istio is configured and ready: + +```bash +kubectl get istio -n gateway-system + +#sample output +# NAME REVISIONS READY IN USE ACTIVE REVISION VERSION AGE +# default 1 1 1 Healthy v1.23.0 3d22h +``` + + + +At this point Kuadrant is installed and ready to be used as is Istio as the gateway provider. This means AuthPolicy and RateLimitPolicy can now be configured and used to protect any Gateways you create. + + +## Configure a DNS provider credentials and a Certificate Issuer + +In this section will build on the previous steps and expand the `kustomization.yaml` we created in the previous step. + +In order for cert-manager and the Kuadrant DNS operator to be able to access and manage DNS records and setup TLS certificates and provide external connectivity for your endpoints, you need to setup a credential for these components. To do this, we will use a Kubernetes secret via a kustomize secret generator. You can find other example overlays for each supported cloud provider under the [configure directory](https://github.com/Kuadrant/kuadrant-operator/tree/main/config/install/configure). + +An example lets-encrypt certificate issuer is provided, but for more information on certificate issuers take a look at the [cert-manager documentation](https://cert-manager.io/docs/configuration/acme/). + + +Lets modify our existing local kustomize overlay to setup these secrets and the cluster certificate issuer: + +First you will need to setup the required `.env` file specified in the kuztomization.yaml file in the same directory as your existing configure kustomization. Below is an example for AWS: + +```bash +touch kudarant/configure/aws-credentials.env ``` -kubectl apply -k config/install/configure/aws +Add the following to your new file + ``` +KUADRANT_AWS_ACCESS_KEY_ID=xxx +KUADRANT_AWS_SECRET_ACCESS_KEY=xxx +KUADRANT_AWS_REGION=eu-west-1 -This will configure Kuadrant and Sail to install their components, set the credentials needed to access DNS zones in the cloud provider, and create a Let's Encrypt cluster issuer configured to use DNS-based validation. +``` -### Validate +With this setup, lets update our configure kustomization and also define a TLS clusterissuer:: + +```yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/configure/standard?ref=v1.0.1 #change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + - cluster-issuer.yaml #(comment if you dont want to use it. The issuer yaml is defined below). Ensure you name the file correctly. + + +generatorOptions: + disableNameSuffixHash: true + labels: + app.kubernetes.io/part-of: kuadrant + app.kubernetes.io/managed-by: kustomize + +secretGenerator: + - name: aws-provider-credentials + namespace: cert-manager # assumes cert-manager namespace exists. + envs: + - aws-credentials.env # notice this matches the .env file above. You will need to setup this file locally + type: 'kuadrant.io/aws' + - name: aws-provider-credentials + namespace: gateway-system # this is the namespace where your gateway will be provisioned + envs: + - aws-credentials.env #notice this matches the .env file above. you need to set up this file locally first. + type: 'kuadrant.io/aws' + + +``` -Validate Kuadrant is ready via the kuadrant resource status condition +Example Lets-Encrypt Cluster Issuer that uses the aws credential. Create this in the same directory as the configure kustomization.yaml: ```bash -kubectl get kuadrant kuadrant -n kuadrant-system -o=yaml +touch kuadrant/configure/cluster-issuer.yaml ``` -At this point Kuadrant is ready to use. Below are some additional configuration that can be applied. +Add the following to this new file: + +```yaml +# example lets-encrypt cluster issuer that will work with the credentials we will add +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: lets-encrypt-aws +spec: + acme: + privateKeySecretRef: + name: le-secret + server: https://acme-v02.api.letsencrypt.org/directory + solvers: + - dns01: + route53: + accessKeyIDSecretRef: + key: AWS_ACCESS_KEY_ID + name: aws-provider-credentials #notice this matches the name of the secret we created. + region: us-east-1 #override if needed + secretAccessKeySecretRef: + key: AWS_SECRET_ACCESS_KEY + name: aws-provider-credentials + +``` + +To configure our installation, re-apply the configure kustomization on the cluster (note this doesn't need to be done in different steps, but is done so here to illustrate how you can build up your configuration of Kuadrant). + +```bash +kubectl apply -k kuadrant/configure +``` + +The cluster issuer should become ready: + +```bash +kubectl get clusterissuer -o=wide + +# NAME READY STATUS AGE +# lets-encrypt-aws True The ACME account was registered with the ACME server 14s + +``` + +We create two credentials. One for use with DNSPolicy in the gateway-system namespace and one for use by cert-manager in the `cert-manager` namespace. With these credentials in place and the cluster issuer configured. You are now ready to start using DNSPolicy and TLSPolicy to secure and connect your Gateways. + + +## Using External Redis + +To connect `Limitador` the component responsible for rate limiting to redis so that its counters are stored and shared with other limitador instances follow these steps: + +Again we will build on the configure kustomization we started. In the same way we did for the cloud provider credentials, we need to setup a `redis-credential.env` file in the same directory as the kustomization. + + +```bash +touch kudarant/configure/redis-credentials.env + +``` + +Add the redis connection string to this file in the following format: + +``` +URL=redis://xxxx +``` + +Next we need to add a new secret generator to our existing configure file at `kuadrant/configure/kustomization.yaml` add it below the other `secretGenerators` + +```yaml + - name: redis-credentials + namespace: kuadrant-system + envs: + - redis-credentials.env + type: 'kuadrant.io/redis' +``` + +We also need to replace the existing limitador resource. Add the following to the `kuadrant/configure` directory. + +```bash +touch kuadrant/configure/limitador.yaml +``` + +Add the following to the `limitador.yaml` file: + +```yaml + +apiVersion: limitador.kuadrant.io/v1alpha1 +kind: Limitador +metadata: + name: limitador + namespace: kuadrant-system +spec: + storage: + redis: + configSecretRef: + name: redis-credentials + +``` + +Add the new resource to your `kuadrant/configure/kustomization.yaml` file under the resources section: + +```yaml +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/configure/standard?ref=v1.0.1 #change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + - cluster-issuer.yaml #(comment if you dont want to use it. The issuer yaml is defined below). + - limitador.yaml # NEW + +``` -### External Redis -create a `redis-credential.env` in the `config/install/configure/redis-storage` dir +Re-Apply the configuration to setup the new secret and limitador configuration: + +>Note you will see a warning here about last applied configuration. This is because we are replacing the resource setup by the kuadrant-operator however this warning wont cause any issues for the installation. ```bash -kubectl apply -k config/install/configure/redis-storage +kubectl apply -k kuadrant/configure/ ``` -This will setup limitador to use provided redis connection URL as a backend store for ratelimit counters. Limitador will becomes temporarily unavailable as it restarts. +Limitador is now configured to use the provided redis connection URL as a backend store for rate limit counters. Limitador will becomes temporarily unavailable as it restarts. ### Validate Validate Kuadrant is in a ready state as before: ```bash -kubectl get kuadrant kuadrant -n kuadrant-system -o=yaml +kubectl get kuadrant kuadrant -n kuadrant-system -o=wide + +# NAME STATUS AGE +# kuadrant Ready 61m + ``` -## Set up observability +## Set up observability (OpenShift Only) Verify that user workload monitoring is enabled in your Openshift cluster. If it not enabled, check the [Openshift documentation](https://docs.openshift.com/container-platform/4.17/observability/monitoring/enabling-monitoring-for-user-defined-projects.html) for how to do this. @@ -110,7 +335,8 @@ kubectl get configmap cluster-monitoring-config -n openshift-monitoring -o jsonp Install the gateway & Kuadrant metrics components and configuration, including Grafana. ```bash -kubectl apply -k config/install/configure/observability +# change the version as needed +kubectl apply -k https://github.com/Kuadrant/kuadrant-operator//config/install/configure/observability?ref=v1.0.1 ``` Configure the Openshift thanos-query instance as a data source in Grafana. @@ -126,7 +352,7 @@ kubectl apply -k config/observability/openshift/grafana Create the example dashboards in Grafana ```bash -kubectl apply -k examples/dashboards +kubectl apply -k https://github.com/Kuadrant/kuadrant-operator//examples/dashboards?ref=v1.0.1 ``` Access the Grafana UI, using the default user/pass of root/secret. @@ -136,3 +362,8 @@ For more information on the example dashboards, check out the [documentation](ht ```bash kubectl -n monitoring get routes grafana-route -o jsonpath="https://{.status.ingress[].host}" ``` + + +### Next Steps + +- Try out one of our user-guides [secure, connect protect](https://docs.kuadrant.io/latest/kuadrant-operator/doc/user-guides/full-walkthrough/secure-protect-connect-k8s/#overview) diff --git a/config/install/configure/aws/kustomization.yaml b/config/install/configure/aws/kustomization.yaml index 831238e72..c7cff08ad 100644 --- a/config/install/configure/aws/kustomization.yaml +++ b/config/install/configure/aws/kustomization.yaml @@ -4,7 +4,8 @@ kind: Kustomization ## NOTE YOU NEED TO CREATE A aws-provider-credentials.env file first! resources: - - ../standard + - https://github.com/Kuadrant/kuadrant-operator//config/install/standard + # - https://github.com/Kuadrant/kuadrant-operator//config/install/openshift # use this one if installing on OCP - cluster-issuer.yaml generatorOptions: @@ -17,7 +18,7 @@ secretGenerator: - name: aws-provider-credentials namespace: cert-manager envs: - - aws-credentials.env + - aws-credentials.env # you will need to setup this file locally type: 'kuadrant.io/aws' - name: aws-provider-credentials namespace: gateway-system diff --git a/config/install/configure/kitchen-sink/kustomization.yaml b/config/install/configure/kitchen-sink/kustomization.yaml deleted file mode 100644 index 74bde7691..000000000 --- a/config/install/configure/kitchen-sink/kustomization.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -resources: - - ../dns-aws - - ../dns-azure - - ../dns-gcp - - ../redis-storage - - ../tls-lets-encrypt diff --git a/config/install/configure/redis-storage/kustomization.yaml b/config/install/configure/redis-storage/kustomization.yaml index 862321f8f..f1f3e6abc 100644 --- a/config/install/configure/redis-storage/kustomization.yaml +++ b/config/install/configure/redis-storage/kustomization.yaml @@ -14,4 +14,4 @@ secretGenerator: namespace: kuadrant-system envs: - redis-credentials.env - type: 'kuadrant.io/aws' + type: 'kuadrant.io/redis' diff --git a/config/install/full-example-aws/configure/aws-credentials.env.sample b/config/install/full-example-aws/configure/aws-credentials.env.sample new file mode 100644 index 000000000..1eb0cc908 --- /dev/null +++ b/config/install/full-example-aws/configure/aws-credentials.env.sample @@ -0,0 +1,5 @@ +# If using this you will need to rename the file and remove the .sample suffix and remove this comment + +KUADRANT_AWS_ACCESS_KEY_ID=xxx +KUADRANT_AWS_SECRET_ACCESS_KEY=xxx +KUADRANT_AWS_REGION=eu-west-1 diff --git a/config/install/full-example-aws/configure/cluster-issuer.yaml b/config/install/full-example-aws/configure/cluster-issuer.yaml new file mode 100644 index 000000000..b6f95e5f0 --- /dev/null +++ b/config/install/full-example-aws/configure/cluster-issuer.yaml @@ -0,0 +1,20 @@ +# example lets-encrypt cluster issuer that will work with the credentials we will add +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: lets-encrypt-aws +spec: + acme: + privateKeySecretRef: + name: le-secret + server: https://acme-v02.api.letsencrypt.org/directory + solvers: + - dns01: + route53: + accessKeyIDSecretRef: + key: AWS_ACCESS_KEY_ID + name: aws-provider-credentials #notice this matches the name of the secret we created. + region: us-east-1 #override if needed + secretAccessKeySecretRef: + key: AWS_SECRET_ACCESS_KEY + name: aws-provider-credentials diff --git a/config/install/full-example-aws/configure/kustomization.yaml b/config/install/full-example-aws/configure/kustomization.yaml new file mode 100644 index 000000000..b3979f665 --- /dev/null +++ b/config/install/full-example-aws/configure/kustomization.yaml @@ -0,0 +1,29 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/configure/standard?ref=v1.0.1 #change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + - cluster-issuer.yaml #(comment if you dont want to use it. The issuer yaml is defined below). + - limitador.yaml + +generatorOptions: + disableNameSuffixHash: true + labels: + app.kubernetes.io/part-of: kuadrant + app.kubernetes.io/managed-by: kustomize + +secretGenerator: + - name: aws-provider-credentials + namespace: cert-manager # assumes cert-manager namespace exists. + envs: + - aws-credentials.env # notice this matches the .env file above. You will need to setup this file locally + type: 'kuadrant.io/aws' + - name: aws-provider-credentials + namespace: gateway-system # this is the namespace where your gateway will be provisioned + envs: + - aws-credentials.env #notice this matches the .env file above. you need to set up this file locally first. + type: 'kuadrant.io/aws' + - name: redis-credentials + namespace: kuadrant-system + envs: + - redis-credentials.env + type: 'kuadrant.io/redis' diff --git a/config/install/full-example-aws/configure/limitador.yaml b/config/install/full-example-aws/configure/limitador.yaml new file mode 100644 index 000000000..a5a3c2e2d --- /dev/null +++ b/config/install/full-example-aws/configure/limitador.yaml @@ -0,0 +1,11 @@ +apiVersion: limitador.kuadrant.io/v1alpha1 +kind: Limitador +metadata: + name: limitador + namespace: kuadrant-system +spec: + storage: + redis: + configSecretRef: + name: redis-credentials + diff --git a/config/install/full-example-aws/configure/redis-credentials.env.sample b/config/install/full-example-aws/configure/redis-credentials.env.sample new file mode 100644 index 000000000..c874708e3 --- /dev/null +++ b/config/install/full-example-aws/configure/redis-credentials.env.sample @@ -0,0 +1,3 @@ +# If using this you will need to rename the file and remove the .sample suffix and remove this comment + +URL=redis://xxxx diff --git a/config/install/full-example-aws/install/kustomization.yaml b/config/install/full-example-aws/install/kustomization.yaml new file mode 100644 index 000000000..37bce8e21 --- /dev/null +++ b/config/install/full-example-aws/install/kustomization.yaml @@ -0,0 +1,17 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - https://github.com/Kuadrant/kuadrant-operator//config/install/standard?ref=v1.0.1 #change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases) + #- https://github.com/Kuadrant/kuadrant-operator//config/install/openshift?ref=v1.0.1 #use if targeting an OCP cluster. Change this version as needed (see https://github.com/Kuadrant/kuadrant-operator/releases). + +patches: # remove the subscription patch if you are installing a development version. It will then use the "preview" channel + - patch: |- + apiVersion: operators.coreos.com/v1alpha1 + kind: Subscription + metadata: + name: kuadrant + spec: + source: kuadrant-operator-catalog + sourceNamespace: kuadrant-system + name: kuadrant-operator + channel: 'stable' #set to preview if not using a release (for example if using main) diff --git a/config/install/no-gateway-provider/kustomization.yaml b/config/install/no-gateway-provider/kustomization.yaml new file mode 100644 index 000000000..f8154c297 --- /dev/null +++ b/config/install/no-gateway-provider/kustomization.yaml @@ -0,0 +1,9 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# note by default this will install the version of kuadrant you checkout from v1 onwards +# if you want to install a different released version you can use the patch options below. +# If you want to install the latest development tag, remove the subscription patch +resources: + - https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml + - ../../deploy/olm diff --git a/config/install/standard/kuadrant-version.yaml b/config/install/standard/kuadrant-version.yaml deleted file mode 100644 index 8eba0cb4b..000000000 --- a/config/install/standard/kuadrant-version.yaml +++ /dev/null @@ -1,6 +0,0 @@ -apiVersion: operators.coreos.com/v1alpha1 -kind: CatalogSource -metadata: - name: kuadrant-operator-catalog -spec: - image: quay.io/kuadrant/kuadrant-operator-catalog:latest #change this to the version you want to install diff --git a/config/install/standard/kustomization.yaml b/config/install/standard/kustomization.yaml index 69d773e2e..f2fe5a105 100644 --- a/config/install/standard/kustomization.yaml +++ b/config/install/standard/kustomization.yaml @@ -2,15 +2,9 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization # note by default this will install the version of kuadrant you checkout from v1 onwards -# if you want to install a different version you can use the patch option below +# if you want to install a different released version you can use the patch options below. +# If you want to install the latest development tag, remove the subscription patch resources: - https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml - sail-operator.yaml - ../../deploy/olm -# patches: -# - target: -# group: operators.coreos.com -# version: v1alpha1 -# kind: CatalogSource -# name: kuadrant-operator-catalog -# path: kuadrant-version.yaml