You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When a tests failed within the last 5 test runs, the test is not marked as flaky. None of the conditions specified in the documentations mark a test as flaky (https://allurereport.org/docs/test-stability/)
To Reproduce
Steps to reproduce the behavior:
Run a test and make sure it passes
Generate allure report
Clean out allure-results folder
Run a test and make sure it fails
Copy history folder from allure-report to allure-results
Generate allure report
Clean out allure-results folder
Run a test and make sure it passes
Copy history folder from allure-report to allure-results
Generate allure report
Repeat steps 7-10 until you have a total of 5 test runs
Expected behavior
Expected the test that failed in the last 5 test runs to be marked as flaky.
Screenshots
This is what my trend looks like where multiple tests failed within the last 5 test runs
But none of the tests were marked as flaky
Desktop :
OS: Windows 11
Browser: Chrome
Version: 131.0.6778.86
Additional context
The tests are also set to retry 3 times when failed
After the tests have completed, it does show in the CLI that there are flaky tests, but it is not shown in the allure-report.
My playwright configuration is also set to only allure-playwright as the reporter
Command used to run tests: npx playwright test JobLogin.spec.ts -g "Job login and logout"
Command used to generate reports: ** allure generate --clean**
The text was updated successfully, but these errors were encountered:
PatatN7
changed the title
Allure-Report not marking tests as flaky (allure-playwright)
[v3.0.5]Allure-Report not marking tests as flaky (allure-playwright)
Nov 28, 2024
Describe the bug
When a tests failed within the last 5 test runs, the test is not marked as flaky. None of the conditions specified in the documentations mark a test as flaky (https://allurereport.org/docs/test-stability/)
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Expected the test that failed in the last 5 test runs to be marked as flaky.
Screenshots
This is what my trend looks like where multiple tests failed within the last 5 test runs
But none of the tests were marked as flaky
Desktop :
Additional context
The tests are also set to retry 3 times when failed
After the tests have completed, it does show in the CLI that there are flaky tests, but it is not shown in the allure-report.
My playwright configuration is also set to only allure-playwright as the reporter
Command used to run tests: npx playwright test JobLogin.spec.ts -g "Job login and logout"
Command used to generate reports: ** allure generate --clean**
The text was updated successfully, but these errors were encountered: