Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent pa11y from silently failing in CI #3780

Open
echappen opened this issue Oct 25, 2023 · 0 comments
Open

Prevent pa11y from silently failing in CI #3780

echappen opened this issue Oct 25, 2023 · 0 comments

Comments

@echappen
Copy link
Contributor

Feature suggestion

When pa11y scans a URL and returns a scanning error, the CI job fails. But when pa11y fails to scan any urls and returns a generic Failed to run error, the CI job still succeeds.

I propose failing the entire CI job in this scenario, in order to prevent silent failures.

User story

As a developer, I want to know if my build is failing so I can prevent a11y bugs from entering production.

Other considerations and design implementation suggestions

Note that existing behavior for pa11y job in CI (before and after the recent pa11y bug) is to SIGHUP regardless if the URL scans succeed. I'm not sure if this behavior was intended or correct, but a SIGHUP may be causing the job to succeed in all scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant