Functional tests for Gradle Plugin not running

[marcospereira]

tl;dr

Gradle Plugin functional tests are not running. I'm not sure if they are still useful, or if the plugin is tested somewhere else.

---

It seems like the functionalTest task for the diktat-gradle-plugin subproject is not working/running (--console verbose :

./gradlew clean :diktat-gradle-plugin:build --console verbose

Outputs (redacted for brevity):

diktat (master *$=) % ./gradlew clean :diktat-gradle-plugin:build --console verbose

> Task :diktat-gradle-plugin:clean
> Task :diktat-gradle-plugin:checkKotlinGradlePluginConfigurationErrors
> Task :diktat-gradle-plugin:pluginDescriptors
> Task :diktat-gradle-plugin:processResources

> Task :diktat-gradle-plugin:detekt

> Task :diktat-gradle-plugin:generateJacocoTestKitProperties
> Task :diktat-gradle-plugin:processTestResources NO-SOURCE

> Task :diktat-gradle-plugin:compileKotlin
w: Language version 1.4 is deprecated and its support will be removed in a future version of Kotlin

> Task :diktat-gradle-plugin:compileJava NO-SOURCE
> Task :diktat-gradle-plugin:classes
> Task :diktat-gradle-plugin:jar
> Task :diktat-gradle-plugin:javadoc NO-SOURCE
> Task :diktat-gradle-plugin:javadocJar
> Task :diktat-gradle-plugin:assemble
> Task :diktat-gradle-plugin:pluginUnderTestMetadata

> Task :diktat-gradle-plugin:compileTestKotlin
w: Language version 1.4 is deprecated and its support will be removed in a future version of Kotlin

> Task :diktat-gradle-plugin:validatePlugins
> Task :diktat-gradle-plugin:compileTestJava NO-SOURCE
> Task :diktat-gradle-plugin:testClasses UP-TO-DATE
> Task :diktat-gradle-plugin:test
> Task :diktat-gradle-plugin:jacocoTestReport
> Task :diktat-gradle-plugin:check
> Task :diktat-gradle-plugin:build

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 13s
55 actionable tasks: 46 executed, 9 up-to-date

And it looks like those tests are broken. If I run:

./gradlew clean :diktat-gradle-plugin:functionaTest

diktat (master *$=) % ./gradlew clean :diktat-gradle-plugin:functionaTest
Type-safe project accessors is an incubating feature.

> Configure project :
git tree is not clean; Untracked files: [], uncommitted changes: [diktat-rules/src/test/kotlin/com/saveourtool/diktat/ruleset/chapter2/KdocCommentsFixTest.kt, diktat-rules/src/test/kotlin/com/saveourtool/diktat/ruleset/chapter1/PackageNamingFixTest.kt, diktat-cli/src/test/kotlin/com/saveourtool/diktat/smoke/DiktatCliTest.kt]
Skipping Nexus publishing configuration as either sonatypeUsername or sonatypePassword are not set
Reckoned version: 2.1.0-rc.0.12+20240111T014759Z

> Task :diktat-gradle-plugin:compileKotlin
w: Language version 1.4 is deprecated and its support will be removed in a future version of Kotlin

> Task :diktat-gradle-plugin:compileFunctionalTestKotlin
w: Language version 1.4 is deprecated and its support will be removed in a future version of Kotlin

> Task :diktat-gradle-plugin:functionalTest

DiktatGradlePluginGroovyFunctionalTest > should execute diktatCheck with explicit configuration() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginGroovyFunctionalTest.kt:50

DiktatGradlePluginFunctionalTest > should execute diktatCheck with gradle older than 6_4() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:129

DiktatGradlePluginFunctionalTest > should have json reporter files() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:56

DiktatGradlePluginFunctionalTest > should execute diktatCheck with gradle older than 6_4() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:129

DiktatGradlePluginGroovyFunctionalTest > should execute diktatCheck with explicit configuration() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginGroovyFunctionalTest.kt:50

DiktatGradlePluginFunctionalTest > should have json reporter files() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:56

DiktatGradlePluginFunctionalTest > should execute diktatCheck with gradle older than 6_4() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:129

DiktatGradlePluginGroovyFunctionalTest > should execute diktatCheck with explicit configuration() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginGroovyFunctionalTest.kt:50

DiktatGradlePluginFunctionalTest > should have json reporter files() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:56

DiktatGradlePluginFunctionalTest > should execute diktatCheck with gradle older than 6_4() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:129

DiktatGradlePluginGroovyFunctionalTest > should execute diktatCheck with explicit configuration() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginGroovyFunctionalTest.kt:50

DiktatGradlePluginFunctionalTest > should have json reporter files() FAILED
    java.lang.IllegalArgumentException at DiktatGradlePluginFunctionalTest.kt:56

19 tests completed, 12 failed

> Task :diktat-gradle-plugin:functionalTest FAILED
> Task :diktat-gradle-plugin:generateJacocoTestKitProperties FAILED

> Task :diktat-gradle-plugin:compileTestKotlin
w: Language version 1.4 is deprecated and its support will be removed in a future version of Kotlin

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':diktat-gradle-plugin:functionalTest'.
> There were failing tests. See the report at: file:///Users/marcospereira/src/github.com/saveourtool/diktat/diktat-gradle-plugin/build/reports/tests/functionalTest/index.html
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
A problem was found with the configuration of task ':diktat-gradle-plugin:generateJacocoTestKitProperties' (type 'WriteProperties').
  - Gradle detected a problem with the following location: '/Users/marcospereira/src/github.com/saveourtool/diktat/diktat-gradle-plugin/build/testkit/test/testkit-gradle.properties'.

    Reason: Task ':diktat-gradle-plugin:functionalTest' uses this output of task ':diktat-gradle-plugin:generateJacocoTestKitProperties' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':diktat-gradle-plugin:generateJacocoTestKitProperties' as an input of ':diktat-gradle-plugin:functionalTest'.
      2. Declare an explicit dependency on ':diktat-gradle-plugin:generateJacocoTestKitProperties' from ':diktat-gradle-plugin:functionalTest' using Task#dependsOn.
      3. Declare an explicit dependency on ':diktat-gradle-plugin:generateJacocoTestKitProperties' from ':diktat-gradle-plugin:functionalTest' using Task#mustRunAfter.

    For more information, please refer to https://docs.gradle.org/8.5/userguide/validation_problems.html#implicit_dependency in the Gradle documentation.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 1m 58s
51 actionable tasks: 38 executed, 13 up-to-date

It looks like the task is not even configured properly regarding Jacoco tasks. I see #1579, when those tests were re-enabled, but that is ancient (when the project was using Maven).

Are these tests still useful? Or is the plugin tested somewhere else? If so, where?

[nulls]

thanks @marcospereira, will look into it.

Tests should be run, probably we broke somewhere it