Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: test one package at a time #2509

Merged
merged 1 commit into from
Jul 21, 2024
Merged

ci: test one package at a time #2509

merged 1 commit into from
Jul 21, 2024

Conversation

tsahee
Copy link
Contributor

@tsahee tsahee commented Jul 19, 2024

This averts a bug where gotesum seems to succeed if there were failures in multiple packages, some packages were o.k. after rerunning failures and some were not.

not sure if this is the best way - but it seems to work.

Here is an example of the problem: see how Go Test(default) step passes CI, and it's "run tests without race detection" step appears as passing even though it did record errors:
https://github.com/OffchainLabs/nitro/actions/runs/10015286108/job/27686456877

Here is execution for the same ci with said fix - notice that both (defaults) and (race) fail - correctly, when executed 3 times:
https://github.com/OffchainLabs/nitro/actions/runs/10014238547?pr=2504

@cla-bot cla-bot bot added the s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA. label Jul 19, 2024
@PlasmaPower
Copy link
Collaborator

I think we should write a shell script, perhaps in .github/workflows or perhaps written to disk in a previous ci step, that does the for package in go list ./...; ... work and the stdbuf work to just accept a list of gotestsum options. That said, I'd also be fine merging this as-is, given that it fixes an issue that otherwise allows CI to pass for failing tests.

Copy link
Collaborator

@PlasmaPower PlasmaPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for now, though I'd still like to refactor this soonish

@tsahee tsahee merged commit 2dbd81c into master Jul 21, 2024
15 checks passed
@tsahee tsahee deleted the ci_one_package_at_a_time branch July 21, 2024 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design-approved s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants