Skip to content

Commit

Permalink
Document PR Automation Testing (#304)
Browse files Browse the repository at this point in the history
Addin docs for pr automation testing strategies.
  • Loading branch information
michaeljguarino authored Sep 21, 2024
1 parent 15679da commit 4860279
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 0 deletions.
58 changes: 58 additions & 0 deletions pages/deployments/pr/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: PR Automation Testing
description: Tools and Strategies for Testing your PR Automations
---

Testing template code is always a bit of a challenge. In the case of a Plural PR automation, you can always experiment using our UI, but if you need more thorough QA we provide two testing strategies:

* local execution with `plural pr test`
* contract testing with `plural pr contracts`

Both can be used, the former is nice for lightweight validation of the correctness of your PR automation, the latter provides a robust CI strategy for more complex setups.

## Locally Execute a PR Automation

You can run a pr automation locally at any time with:

```sh
plural pr test --file path/to/pr-automation-cr.yaml
```

It will parse the CRD locally, run a terminal-based wizard for its configuration, and apply the templates using the same codepath executed by the main PR automation api. No PR will actually be created, but you can see exactly what the templates will do, and confirm it works as expected.

## Contract Testing a PR Automation

Contract testing is a test strategy which takes the output of a set of APIs against expected inputs, renders it locally into your source control, and succeeds only if there are no differences created. This provides a clear way to know if a prior contract was violated by showing the drift created, and integrates seamlessly into a code review + CI process, since you can execute it in your CI solution and view the changes in a PR diff.

The `plural` CLI has a command that easily wraps this approach with a single declarative file. You'll create yaml file like this, say in a file named `test/contracts.yaml`:

```yaml
apiVersion: platform.plural.sh/v1alpha1
kind: PrContracts
metadata:
name: workspaces
spec:
workdir: outputs # where to execute pr automations, useful if you want to run inside a test folder
automations:
- file: path/to/pr-automation.yaml
context: ../contexts/pr-automation-context.yaml # yaml file representing the configuration input for this automation
- file: path/to/other-automation.yaml
context: ../contexts/other-automation-context.yaml
- file: path/to/external-automation.yaml
context: ../contexts/pr-automation-context.yaml
externalDir: other-templates # if you're using a PR automation sourcing templates from an external git reference, you need to configure an `externalDir` pointing to the templates it would have pulled in.
```
If you want to wrap this all up into a script executable by your CI, you can imitate this [Justfile](https://just.systems/man/en/)
```make
contracts:
cd test && plural pr contracts --file contracts.yaml

test: contracts
if [[ `git status --porcelain` ]]; then \
echo "local git changes detected, failing contract test"; \
git --no-pager diff; \
exit 1; \
fi
```
4 changes: 4 additions & 0 deletions src/NavData.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,10 @@ const rootNavData: NavMenu = deepFreeze([
title: 'On Demand Pull Requests',
href: '/deployments/pr/crds',
},
{
title: "Testing PR Automations",
href: '/deployments/pr/testing',
},
{
title: 'Pull Request Pipelines',
href: '/deployments/pr/pipelines',
Expand Down

0 comments on commit 4860279

Please sign in to comment.