Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When test retry fails, it obscures the actual failed assertion #351

Open
adam-enko opened this issue Nov 12, 2024 · 0 comments
Open

When test retry fails, it obscures the actual failed assertion #351

adam-enko opened this issue Nov 12, 2024 · 0 comments

Comments

@adam-enko
Copy link

adam-enko commented Nov 12, 2024

Test-retry is enabled in Dokka.

https://github.com/Kotlin/dokka/blob/df672b74de0d47679ba7c73b331de83de3608623/build-logic/src/main/kotlin/dokkabuild.base.gradle.kts#L27-L32

When a test fails, test-retry attempts to retry it, but fails (probably because some Dokka tests use Kotest #93).

Aside from the Kotest compatibility issues, it's inconvenient that the actual test error has been obscured.

All I can see in the build logs is the location of the failed assertion, but I can't see any of the error message.

For example, I would expect to see the withClue message: "files with errors: ${filesWithErrors.joinToString("\n") { " - ${it.toUri()}" }}"

https://github.com/Kotlin/dokka/blob/383f029a89a5ab3b58ec59ac1a8477cd0e179ed5/dokka-runners/dokka-gradle-plugin/src/testFunctional/kotlin/KmpCommonSourceSharedWithDependentsTest.kt#L39-L43

> Task :dokka-gradle-plugin:test FROM-CACHE
> Task :dokka-gradle-plugin:testFunctional

org.jetbrains.dokka.gradle.KmpCommonSourceSharedWithDependentsTest > common source set is propagated to dependents FAILED
    java.lang.AssertionError at KmpCommonSourceSharedWithDependentsTest.kt:42
        Caused by: java.lang.AssertionError at KmpCommonSourceSharedWithDependentsTest.kt:42

105 tests completed, 1 failed
There were failing tests. See the report at: file:///home/runner/work/dokka/dokka/dokka-runners/dokka-gradle-plugin/build/reports/tests/testFunctional/index.html

> Task :dokka-gradle-plugin:testFunctional FAILED

342 actionable tasks: 231 executed, 111 from cache
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dokka-gradle-plugin:testFunctional'.
> The following test methods could not be retried, which is unexpected. Please file a bug report at https://github.com/gradle/test-retry-gradle-plugin/issues
     org.jetbrains.dokka.gradle.KmpCommonSourceSharedWithDependentsTest#common source set is propagated to dependents


* Try:
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org./

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dokka-gradle-plugin:testFunctional'.
...


BUILD FAILED in 9m 16s

Publishing build scan...
https://ge.jetbrains.com/s/jhbii4zkbocee

Build scan: https://ge.jetbrains.com/s/jhbii4zkbocee

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant