Skip to content

Commit

Permalink
added e2e tests (#96)
Browse files Browse the repository at this point in the history
Signed-off-by: Ved Ratan <[email protected]>
  • Loading branch information
VedRatan authored May 2, 2024
1 parent b8330d5 commit 0f69a67
Show file tree
Hide file tree
Showing 17 changed files with 808 additions and 6 deletions.
99 changes: 96 additions & 3 deletions .github/workflows/pr-checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ jobs:
working-directory: ./pkg/adapter/${{ matrix.adapters }}
run: make docker-build

chainsaw-tests:
name: Test
chainsaw-integration-tests:
name: Integration-Test
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down Expand Up @@ -128,4 +128,97 @@ jobs:
kubectl get pods -A
- name: Run Tests
run: make test
run: make integration-test

chainsaw-e2e-tests:
name: E2E-Test
runs-on: ubuntu-latest
steps:
- name: Checkout source code
uses: actions/checkout@v3

- name: Install helm
id: helm
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3.5
with:
token: ${{ secrets.GITHUB_TOKEN }}

- name: Create k8s Kind Cluster
uses: helm/kind-action@v1
with:
cluster_name: testing

- name: Build nimbus image and load in the kind cluster
run: |
make docker-build
kind load docker-image 5gsec/nimbus:latest --name=testing
- name: Build nimbus-netpol image and load in the kind cluster
working-directory: ./pkg/adapter/nimbus-netpol
run: |
make docker-build
kind load docker-image 5gsec/nimbus-netpol:latest --name=testing
- name: Build nimbus-kubearmor image and load in the kind cluster
working-directory: ./pkg/adapter/nimbus-kubearmor
run: |
make docker-build
kind load docker-image 5gsec/nimbus-kubearmor:latest --name=testing
- name: Build nimbus-kyverno image and load in the kind cluster
working-directory: ./pkg/adapter/nimbus-kyverno
run: |
make docker-build
kind load docker-image 5gsec/nimbus-kyverno:latest --name=testing
- name: Install Kubearmor CRDs
run: |
kubectl create -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/deployments/CRD/KubeArmorPolicy.yaml
- name: Install Kyverno CRDs
run: |
kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/config/crds/kyverno/kyverno.io_clusterpolicies.yaml
kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/config/crds/kyverno/kyverno.io_policies.yaml

- name: Install Nimbus
run: |
helm upgrade --install nimbus-operator deployments/nimbus -n nimbus --create-namespace --set image.pullPolicy=Never
- name: Wait for Nimbus to start
run: |
kubectl wait --for=condition=ready --timeout=5m -n nimbus pod -l app.kubernetes.io/name=nimbus
kubectl get pods -A
- name: Install nimbus-netpol
working-directory: deployments/nimbus-netpol/
run: |
helm upgrade --install nimbus-netpol . -n nimbus --set image.pullPolicy=Never
- name: Wait for nimbus-netpol to start
run: |
kubectl wait --for=condition=ready --timeout=5m -n nimbus pod -l app.kubernetes.io/name=nimbus-netpol
kubectl get pods -A
- name: Install nimbus-kubearmor
working-directory: deployments/nimbus-kubearmor/
run: |
helm upgrade --install nimbus-kubearmor . -n nimbus --set image.pullPolicy=Never
- name: Wait for nimbus-kubearmor to start
run: |
kubectl wait --for=condition=ready --timeout=5m -n nimbus pod -l app.kubernetes.io/name=nimbus-kubearmor
kubectl get pods -A
- name: Install nimbus-kyverno
working-directory: deployments/nimbus-kyverno/
run: |
helm upgrade --install nimbus-kyverno . -n nimbus --set image.pullPolicy=Never
- name: Wait for nimbus-kyverno to start
run: |
kubectl wait --for=condition=ready --timeout=5m -n nimbus pod -l app.kubernetes.io/name=nimbus-kyverno
kubectl get pods -A
- name: Run Tests
run: make e2e-test
12 changes: 9 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ IMG ?= 5gsec/nimbus
# Image Tag to use all building/pushing image targets
TAG ?= v0.1

TEST_DIR ?= tests/controllers

# Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set)
ifeq (,$(shell go env GOBIN))
GOBIN=$(shell go env GOPATH)/bin
Expand Down Expand Up @@ -62,13 +64,17 @@ fmt: ## Run go fmt against code.
vet: ## Run go vet against code.
go vet ./...

.PHONY: test
test: chainsaw ## Run tests.
.PHONY: integration-test
integration-test: chainsaw ## Run integration tests.
@$(LOCALBIN)/chainsaw test --test-dir=tests/controllers/ --config tests/chainsaw-config.yaml

.PHONY: e2e-test
e2e-test: chainsaw ## Run e2e tests.
@$(LOCALBIN)/chainsaw test --test-dir=tests/e2e/ --config tests/chainsaw-config.yaml

.PHONY: test-doc
test-doc: chainsaw ## Build documentation of tests.
@$(LOCALBIN)/chainsaw build docs --test-dir=tests/controllers/
@$(LOCALBIN)/chainsaw build docs --test-dir=$(TEST_DIR)

GOLANGCI_LINT = $(shell pwd)/bin/golangci-lint
GOLANGCI_LINT_VERSION ?= v1.54.2
Expand Down
112 changes: 112 additions & 0 deletions tests/e2e/dns-manipulation/create/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Test: `netpol-kubearmor-adapter-policy-creation`

This test validates that creating a `dns-manipulation` SecurityIntent with SecurityIntentBinding generates the expected Network Policy and KubeArmor policy.


## Steps

| # | Name | Bindings | Try | Catch | Finally |
|:-:|---|:-:|:-:|:-:|:-:|
| 1 | [Create a SecurityIntent](#step-Create a SecurityIntent) | 0 | 1 | 0 | 0 |
| 2 | [Create a SecurityIntentBinding](#step-Create a SecurityIntentBinding) | 0 | 1 | 0 | 0 |
| 3 | [Verify NimbusPolicy creation](#step-Verify NimbusPolicy creation) | 0 | 1 | 0 | 0 |
| 4 | [Verify NetworkPolicy creation](#step-Verify NetworkPolicy creation) | 0 | 1 | 0 | 0 |
| 5 | [Verify KubeArmorPolicy creation](#step-Verify KubeArmorPolicy creation) | 0 | 1 | 0 | 0 |
| 6 | [Verify status of created SecurityIntentBinding](#step-Verify status of created SecurityIntentBinding) | 0 | 1 | 0 | 0 |
| 7 | [Verify status of created NimbusPolicy](#step-Verify status of created NimbusPolicy) | 0 | 1 | 0 | 0 |
| 8 | [Verify that the corresponding netpol native policy is being updated in the NimbusPolicy status](#step-Verify that the corresponding netpol native policy is being updated in the NimbusPolicy status) | 0 | 1 | 0 | 0 |
| 9 | [Verify that the corresponding kubearmor native policy is being updated in the NimbusPolicy status](#step-Verify that the corresponding kubearmor native policy is being updated in the NimbusPolicy status) | 0 | 1 | 0 | 0 |

### Step: `Create a SecurityIntent`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `apply` | 0 | 0 | *No description* |

### Step: `Create a SecurityIntentBinding`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `apply` | 0 | 0 | *No description* |

### Step: `Verify NimbusPolicy creation`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `assert` | 0 | 0 | *No description* |

### Step: `Verify NetworkPolicy creation`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `assert` | 0 | 0 | *No description* |

### Step: `Verify KubeArmorPolicy creation`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `assert` | 0 | 0 | *No description* |

### Step: `Verify status of created SecurityIntentBinding`

Verify the created SecurityIntentBinding status subresource includes the number and names of bound intents, along with the generated NimbusPolicy name.


#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `assert` | 0 | 0 | *No description* |

### Step: `Verify status of created NimbusPolicy`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `assert` | 0 | 0 | *No description* |

### Step: `Verify that the corresponding netpol native policy is being updated in the NimbusPolicy status`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `script` | 0 | 0 | *No description* |

### Step: `Verify that the corresponding kubearmor native policy is being updated in the NimbusPolicy status`

*No description*

#### Try

| # | Operation | Bindings | Outputs | Description |
|:-:|---|:-:|:-:|---|
| 1 | `script` | 0 | 0 | *No description* |

---

65 changes: 65 additions & 0 deletions tests/e2e/dns-manipulation/create/chainsaw-test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# SPDX-License-Identifier: Apache-2.0
# Copyright 2023 Authors of Nimbus

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: netpol-kubearmor-adapter-policy-creation
spec:
description: >
This test validates that creating a `dns-manipulation` SecurityIntent with SecurityIntentBinding generates the expected Network Policy and KubeArmor policy.
steps:
- name: "Create a SecurityIntent"
try:
- apply:
file: ../../resources/namespaced/dns-manipulation-si.yaml

- name: "Create a SecurityIntentBinding"
try:
- apply:
file: ../../resources/namespaced/dns-manipulation-sib.yaml

- name: "Verify NimbusPolicy creation"
try:
- assert:
file: ../nimbus-policy-assert.yaml

- name: "Verify NetworkPolicy creation"
try:
- assert:
file: ../netpol.yaml

- name: "Verify KubeArmorPolicy creation"
try:
- assert:
file: ../ksp.yaml

- name: "Verify status of created SecurityIntentBinding"
description: >
Verify the created SecurityIntentBinding status subresource includes the number and names of bound intents,
along with the generated NimbusPolicy name.
try:
- assert:
file: ../sib-status-assert.yaml

- name: "Verify status of created NimbusPolicy"
try:
- assert:
file: ../np-status-assert.yaml

- name: "Verify that the corresponding NimbusPolicy status has been updated with the generated Network Policy"
try:
- script:
content: kubectl get np -n $NAMESPACE dns-manipulation-binding -o=jsonpath='{.status.adapterPolicies}'
check:
(contains($stdout, 'NetworkPolicy/dns-manipulation-binding-dnsmanipulation')): true

- name: "Verify that the corresponding NimbusPolicy status has been updated with the generated KubeArmor Policy"
try:
- script:
content: kubectl get np -n $NAMESPACE dns-manipulation-binding -o=jsonpath='{.status.adapterPolicies}'
check:
(contains($stdout, 'KubeArmorPolicy/dns-manipulation-binding-dnsmanipulation')): true



Loading

0 comments on commit 0f69a67

Please sign in to comment.