Skip to content

Commit

Permalink
Initial commit. Create base flow for integration tests with data caterer
Browse files Browse the repository at this point in the history
  • Loading branch information
pflooky committed Jun 24, 2024
1 parent 49d301d commit 4868461
Show file tree
Hide file tree
Showing 19 changed files with 5,889 additions and 139 deletions.
2 changes: 1 addition & 1 deletion CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Repository CODEOWNERS

* @actions/actions-oss-maintainers
* @data-catering
69 changes: 69 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,3 +203,72 @@ steps:
id: output
run: echo "${{ steps.run-action.outputs.time }}"
```
# Automatic Integration Tests via GitHub Actions
1. Create pull request
2. Triggers GitHub action
3. Sets up services via insta-infra
4. Data generated via data-caterer
5. Results published to pull request
# Setup
- Create new GitHub application
- Follows similar flow to renovatebot
- Creates initial pull request with template YAML file and GitHub action YAML
- Can try to detect services used (i.e. check `src/test/resources`)
- Can try detect via dependencies (i.e. build.gradle, pom.xml,
requirements.txt, etc.)
- Users can manually add the services their app interacts with (i.e.
Postgres, MySQL)
- The target repo could be one or more apps/jobs
- Need to know how to run their app/job
- Use docker image by default, otherwise build and start script could be
provided by user
- Use default credentials of each service
- Ability to define custom data generation/validation with YAML

# Example Flow

GitHub Action YAML

```yaml
name: Integration Test
on:
push:
branches:
- *
jobs:
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
steps:
- name: Run integration tests
uses: data-catering/data-caterer-action@v1
```

YAML config file:

```yaml
services: #what external services your app/job connects to
- name: kafka:1.1.0 #optionally define a version
data: https://github.com/data-catering/insta-infra/blob/main/data/kafka/my_data.sh
- name: postgres
data: src/main/resources/postgres/ddl
run: #how to run your app/job, can run multiple, run in order
- command: ./run-app.sh
# command: java -jar build/target/my-app.jar
# command: docker run -p 8080:8080 my-image:${APP_VERSION:-1.3.1} #allow for env variable substitution anywhere in the YAML
env:
- APP_VERSION=1.3.1
test: #using data-caterer, generate and validate data
generation:
- name: kafka #name matches with service from above
validation:
- name: postgres
options: #additional options
keepAlive: true #could allow services to be kept alive after running
deleteData: false #retain data for further investigation/debugging/testing
```
10 changes: 6 additions & 4 deletions __tests__/main.test.js → __tests__/main.xtest.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@ describe('action', () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'milliseconds':
return '500'
case 'configuration-file':
return 'example/sample-config.yaml'
case 'insta-infra-folder':
return '/Users/peter/code/insta-infra'
default:
return ''
}
Expand All @@ -35,7 +37,7 @@ describe('action', () => {
await main.run()
expect(runMock).toHaveReturned()

// Verify that all of the core library functions were called correctly
// Verify that all the core library functions were called correctly
expect(debugMock).toHaveBeenNthCalledWith(1, 'Waiting 500 milliseconds ...')
expect(debugMock).toHaveBeenNthCalledWith(
2,
Expand Down Expand Up @@ -87,7 +89,7 @@ describe('action', () => {
await main.run()
expect(runMock).toHaveReturned()

// Verify that all of the core library functions were called correctly
// Verify that all the core library functions were called correctly
expect(setFailedMock).toHaveBeenNthCalledWith(
1,
'Input required and not supplied: milliseconds'
Expand Down
38 changes: 27 additions & 11 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,35 @@
name: 'The name of your action here'
description: 'Provide a description here'
author: 'Your name or organization here'
name: 'Data Caterer data flow testing'
description: 'Run integration tests for your application or job'
author: 'Data Catering'

# Define your inputs here.
inputs:
milliseconds:
description: 'Your input description here'
required: true
default: '1000'
configuration-file:
description: 'File path to Data Caterer configuration file'
default: 'data-caterer.yaml'
insta-infra-folder:
description: 'Folder to insta-infra'
default: 'data-caterer-integration-test/insta-infra'

# Define your outputs here.
outputs:
time:
description: 'Your output description here'
results:
description: 'Result of integration tests'

runs:
using: node20
main: dist/index.js
using: 'composite'
steps:
- uses: actions/checkout@v4
- name: Checkout insta-infra repo
uses: actions/checkout@v4
with:
repository: data-catering/insta-infra
path: data-caterer-integration-test/insta-infra
- name: Run integration tests
uses: actions/github-script@v7
with:
configuration-file: ${{ inputs.configuration-file }}
insta-infra-folder: ${{ inputs.insta-infra-folder }}
script: |
const script = require("${{github.action_path}}/dist/index.js")
script({core})
2 changes: 1 addition & 1 deletion badges/coverage.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 4868461

Please sign in to comment.