Skip to content

Commit

Permalink
report all files in CI test coverage (#10560)
Browse files Browse the repository at this point in the history
refs: #10559

## Description

#10556 made coverage reports include all files, but only in package reports. This makes CI's report across all packages also use `--all`.

While working on this I wanted to be able to document all the kinks I ran into so I also consolidated this capability in a single script with explanatory comments. 

### Security Considerations
none

### Scaling Considerations
none

### Documentation Considerations
includes docs now

### Testing Considerations
The `coverage` job  runs in the after-merge workflow but only on `push` so I can't trigger it in CI. Instead I've run the script locally. When it reaches cosmic-swingset it dies on `./bin/rosetta-cli` but I just skipped that package and confirmed the generated report is as expected.

### Upgrade Considerations
n/a
  • Loading branch information
mergify[bot] authored Nov 23, 2024
2 parents 62aabb6 + 476f7a5 commit d7a0e3b
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 9 deletions.
6 changes: 2 additions & 4 deletions .github/workflows/after-merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,8 @@ jobs:
with:
node-version: ${{ matrix.node-version }}

- name: generate coverage for all tests
run: 'yarn test:c8-all || :'
- name: generate coverage/html reports
run: mkdir -p coverage/tmp && yarn c8 report --reporter=html-spa --reports-dir=coverage/html --temp-directory=coverage/tmp
- name: generate test coverage report
run: ./scripts/ci/generage-test-coverage-report.sh
- uses: actions/upload-artifact@v4
with:
name: coverage
Expand Down
5 changes: 1 addition & 4 deletions COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,7 @@ source-to-source transforms (such as `@endo/bundle-source`,
Coverage reports for the current main branch are
published by CI to: https://agoric-sdk-coverage.netlify.app

It's made by a CI job calling `test:c8-all` in the project root. That in turn
calls `test:c8` in each package, with `$C8_OPTIONS` set to a common coverage
directory and to leave temp files so they can accumulate. The job then uses that
output in another call to c8 to generate a report.
See `scripts/ci/generate-test-coverage-report.sh`

## Per package
You can create a report in any package:
Expand Down
1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@
"lint": "run-s --continue-on-error lint:*",
"lint:packages": "yarn lerna run --no-bail lint",
"test": "yarn lerna run --no-bail test",
"test:c8-all": "rm -rf coverage/tmp && C8_OPTIONS=\"--clean=false --temp-directory=$PWD/coverage/tmp\" lerna run test:c8",
"test:xs": "yarn workspaces run test:xs",
"build": "yarn workspaces run build && scripts/agd-builder.sh stamp yarn-built",
"postinstall": "patch-package && scripts/agd-builder.sh stamp yarn-installed",
Expand Down
17 changes: 17 additions & 0 deletions scripts/ci/generate-test-coverage-report.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

# CI artifact export expects the files to be under `coverage` (same for /html below)
export NODE_V8_COVERAGE="$PWD/coverage/tmp"

# clear out old coverage. c8 usually does this but we have to clean=false below so it accumulates across packages
rm -rf "$NODE_V8_COVERAGE"
mkdir -p "$NODE_V8_COVERAGE"

# the package test:c8 commands will include this
export C8_OPTIONS="--clean=false"

# XXX uses lerna when `yarn workspaces run` should work, but in v1 it always bails on missing script
yarn lerna run test:c8

# report over all src and tools files, not just the ones that were loaded during tests
yarn c8 report --all --include 'packages/*/{src,tools}' --reporter=html-spa --reports-dir=coverage/html

0 comments on commit d7a0e3b

Please sign in to comment.