Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/AN-4120 playwright e2e tests #93

Merged
merged 40 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
5f68c81
add playwright and delete cypress
MGJamJam Jul 5, 2024
90099bc
add first queryEditor playwright test
MGJamJam Jul 16, 2024
bbea871
Merge branch 'main' into feat/AN-4120_playwright-e2e-tests
MGJamJam Jul 16, 2024
489bfda
check for header and start and end date
MGJamJam Jul 16, 2024
255cc8b
set minuteIntervalLimitInMilliseconds to 3h1min
MGJamJam Jul 19, 2024
3c472c9
add defaultPercentileValue
MGJamJam Jul 19, 2024
3f601ac
add ids for testing to ui elements in Query Editor
MGJamJam Jul 19, 2024
9faa6ef
improve and add more test QueryEditor test cases
MGJamJam Jul 19, 2024
858b01b
update yarn.lock
MGJamJam Jul 19, 2024
35f2660
delete example test file
MGJamJam Jul 19, 2024
3a7a57c
add e2e test github workflow
MGJamJam Jul 19, 2024
5d0230c
delete console.log and comment Firefox and Safari tests
MGJamJam Jul 19, 2024
3f46673
fix package conflicts
MGJamJam Jul 22, 2024
1516e54
fix intervalUtils test
MGJamJam Jul 22, 2024
31e0272
update yarn to latest version
MGJamJam Jul 22, 2024
a9c7620
delete deprecated --prefer-offline flag
MGJamJam Jul 22, 2024
6a070fe
install @typescript-eslint/eslint-plugin
MGJamJam Jul 22, 2024
8962cba
delete enableGlobalCache flag in yarnrc
MGJamJam Jul 22, 2024
fbfe079
Revert "delete enableGlobalCache flag in yarnrc"
MGJamJam Jul 22, 2024
40e8ae8
Revert "install @typescript-eslint/eslint-plugin"
MGJamJam Jul 22, 2024
453e2ac
Revert "delete deprecated --prefer-offline flag"
MGJamJam Jul 22, 2024
c6a8611
Revert "update yarn to latest version"
MGJamJam Jul 22, 2024
028b14e
add resolutions to solve conflicting package versions
MGJamJam Jul 22, 2024
63bad25
delete playwright.yml workflow
MGJamJam Jul 22, 2024
89db892
fix should send correct query for time series and table data
MGJamJam Jul 23, 2024
c1255a0
fix should send correct query for percentile selection
MGJamJam Jul 23, 2024
9921e08
add filter test
MGJamJam Jul 23, 2024
db42354
add groupBy and orderBy tests
MGJamJam Jul 23, 2024
a5b04b5
add queryEditorId to element ids
MGJamJam Jul 23, 2024
9e47d6b
add tests to check if data is correctly displayed
MGJamJam Jul 23, 2024
21bd94e
add test for config editor
MGJamJam Jul 23, 2024
558abcc
deletes commented code in playwright config
MGJamJam Jul 23, 2024
53c5c3a
include e2e workflow in the ci workflow
MGJamJam Jul 24, 2024
f95514e
delete interval and percentile changes as they were addressed in othe…
MGJamJam Jul 25, 2024
17a5001
merge main into feat/AN-4120_playwright-e2e-tests
MGJamJam Jul 25, 2024
5ce366a
format
MGJamJam Jul 25, 2024
5fd2f3e
adds unsuccessfull configEditor scenario
MGJamJam Jul 25, 2024
146e89d
regenerate yarn.lock and delete resolutions in package.json
MGJamJam Jul 25, 2024
2d02409
Revert "regenerate yarn.lock and delete resolutions in package.json"
MGJamJam Jul 25, 2024
eef928e
substitute id with data-testid for elements with HTMLElement Attributes
MGJamJam Jul 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 100 additions & 24 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,17 @@ permissions: read-all

jobs:
build:
name: Build, lint and unit tests
runs-on: ubuntu-latest
outputs:
plugin-id: ${{ steps.metadata.outputs.plugin-id }}
plugin-version: ${{ steps.metadata.outputs.plugin-version }}
has-e2e: ${{ steps.check-for-e2e.outputs.has-e2e }}
has-backend: ${{ steps.check-for-backend.outputs.has-backend }}
env:
GRAFANA_ACCESS_POLICY_TOKEN: ${{ secrets.GRAFANA_ACCESS_POLICY_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}

- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
Expand Down Expand Up @@ -65,31 +68,11 @@ jobs:
- name: Check for E2E
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why changes in github workflow files ? Have they not been generated by grafana plugin generator tool ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See answer below.
But good point, instead of having a seperate e2e workflow I will include this here. I will check how the ci file that is generated today with the tool looks like and copy it over.

id: check-for-e2e
run: |
if [ -d "cypress" ]
if [ -f "playwright.config.ts" ]
then
echo "has-e2e=true" >> $GITHUB_OUTPUT
fi

- name: Start grafana docker
if: steps.check-for-e2e.outputs.has-e2e == 'true'
run: docker-compose up -d

- name: Run e2e tests
if: steps.check-for-e2e.outputs.has-e2e == 'true'
run: yarn run e2e

- name: Stop grafana docker
if: steps.check-for-e2e.outputs.has-e2e == 'true'
run: docker-compose down

- name: Archive E2E output
uses: actions/upload-artifact@v4
if: steps.check-for-e2e.outputs.has-e2e == 'true' && steps.run-e2e-tests.outcome != 'success'
with:
name: cypress-videos
path: cypress/videos
retention-days: 5

- name: Sign plugin
run: yarn run sign
if: ${{ env.GRAFANA_ACCESS_POLICY_TOKEN != '' }}
Expand Down Expand Up @@ -119,3 +102,96 @@ jobs:
name: ${{ steps.metadata.outputs.plugin-id }}-${{ steps.metadata.outputs.plugin-version }}
path: ${{ steps.metadata.outputs.plugin-id }}
retention-days: 5

resolve-versions:
name: Resolve e2e images
runs-on: ubuntu-latest
timeout-minutes: 3
needs: build
if: ${{ needs.build.outputs.has-e2e == 'true' }}
outputs:
matrix: ${{ steps.resolve-versions.outputs.matrix }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Resolve Grafana E2E versions
id: resolve-versions
uses: grafana/plugin-actions/e2e-version@main

playwright-tests:
needs: [resolve-versions, build]
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
GRAFANA_IMAGE: ${{fromJson(needs.resolve-versions.outputs.matrix)}}
name: e2e test ${{ matrix.GRAFANA_IMAGE.name }}@${{ matrix.GRAFANA_IMAGE.VERSION }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Download plugin
uses: actions/download-artifact@v4
with:
path: dist
name: ${{ needs.build.outputs.plugin-id }}-${{ needs.build.outputs.plugin-version }}

- name: Execute permissions on binary
if: needs.build.outputs.has-backend == 'true'
run: |
chmod +x ./dist/gpx_*

- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'yarn'

- name: Install dev dependencies
run: yarn install --immutable --prefer-offline

- name: Start Grafana
run: |
docker-compose pull
DEVELOPMENT=false GRAFANA_VERSION=${{ matrix.GRAFANA_IMAGE.VERSION }} GRAFANA_IMAGE=${{ matrix.GRAFANA_IMAGE.NAME }} docker-compose up -d

- name: Wait for Grafana to start
uses: nev7n/wait_for_response@v1
with:
url: 'http://localhost:3000/'
responseCode: 200
timeout: 60000
interval: 500

- name: Install Playwright Browsers
run: yarn exec playwright install chromium --with-deps

- name: Run Playwright tests
id: run-tests
run: yarn run e2e

- name: Docker logs
if: ${{ always() && steps.run-tests.outcome == 'failure' }}
run: |
docker logs test-test-datasource >& grafana-server.log

- name: Stop grafana docker
run: docker-compose down

- name: Upload server log
uses: actions/upload-artifact@v4
if: ${{ always() && steps.run-tests.outcome == 'failure' }}
with:
name: ${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}-server-log
path: grafana-server.log
retention-days: 5

# Uncomment this step to upload the Playwright report to Github artifacts.
# If your repository is public, the report will be public on the Internet so beware not to expose sensitive information.
# - name: Upload artifacts
# uses: actions/upload-artifact@v4
# if: ${{ always() && steps.run-tests.outcome == 'failure' }}
# with:
# name: playwright-report-${{ matrix.GRAFANA_IMAGE.NAME }}-v${{ matrix.GRAFANA_IMAGE.VERSION }}-${{github.run_id}}
# path: playwright-report/
# retention-days: 5
10 changes: 7 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,15 @@ coverage
artifacts/
work/
ci/
e2e-results/
**/cypress/videos
**/cypress/report.json
Comment on lines -27 to -29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is "cypress" mentioned in project files ? I thought new plugin has only playwright ? is this just left over from angular grafana plugin ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cypress is mentioned because grafana just recently deprecated cypress in their plugin development (https://grafana.com/docs/grafana/latest/breaking-changes/breaking-changes-v11-0/#the-grafanae2e-testing-tool-is-deprecated) and back in March when I created the plugin with the generator tool, this generator tool was still using cypress.


# Editor
.idea

.eslintcache

# Playwright
/test-results/
/playwright-report/
/blob-report/
/playwright/.cache/
/playwright/.auth/
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ This project was generated with `yarn create @grafana/[email protected]`.
yarn run server
```

6. Run the E2E tests (using Cypress)
6. Run the E2E tests (using Playwright)

```bash
# Spins up a Grafana instance first that we tests against
Expand Down
3 changes: 0 additions & 3 deletions cypress.json

This file was deleted.

10 changes: 0 additions & 10 deletions cypress/integration/01-smoke.spec.ts

This file was deleted.

47 changes: 47 additions & 0 deletions e2e/configurationEditor.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import { test, expect } from '@grafana/plugin-e2e';
import { BitmovinDataSourceOptions } from '../src/types/grafanaTypes';

test('should save and test valid configuration', async ({
createDataSourceConfigPage,
readProvisionedDataSource,
page,
selectors,
}) => {
await page.route('*/**/analytics/licenses', async (route) => {
await route.fulfill({ status: 200, body: 'OK' });
});
const ds = await readProvisionedDataSource<BitmovinDataSourceOptions>({ fileName: 'datasources.yml' });
const configPage = await createDataSourceConfigPage({ type: ds.type });

await page.getByTestId(`config-editor-${configPage.datasource.name}_api-key-input`).fill('test-api-key');
await page.getByTestId(`config-editor-${configPage.datasource.name}_tenant-org-id-input`).fill('test-tenant-org-id');

const queryPromise = page.waitForRequest('*/**/analytics/licenses');
await configPage.getByGrafanaSelector(selectors.pages.DataSource.saveAndTest).click();
const queryRequest = await queryPromise;

expect(queryRequest.headers()['x-api-client']).toBe('analytics-grafana-datasource');
expect(queryRequest.headers()['x-api-key']).toBe('test-api-key');
expect(queryRequest.headers()['x-tenant-org-id']).toBe('test-tenant-org-id');

expect(configPage).toHaveAlert('success');
});

test('should not save invalid configuration', async ({
createDataSourceConfigPage,
readProvisionedDataSource,
page,
selectors,
}) => {
const ds = await readProvisionedDataSource<BitmovinDataSourceOptions>({ fileName: 'datasources.yml' });
const configPage = await createDataSourceConfigPage({ type: ds.type });

await page.getByTestId(`config-editor-${configPage.datasource.name}_api-key-input`).fill('grafana-invalid-api-key');

const responsePromise = page.waitForResponse('*/**/analytics/licenses');
await configPage.getByGrafanaSelector(selectors.pages.DataSource.saveAndTest).click();
const response = await responsePromise;

expect(response.status()).toBe(403);
expect(configPage).toHaveAlert('error');
});
Loading
Loading