Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: don't download webapps when only running tests #18801

Merged
merged 5 commits into from
Oct 22, 2024
Merged

Conversation

teleivo
Copy link
Contributor

@teleivo teleivo commented Oct 10, 2024

These are the maven lifecycles which we can execute. We can also bind
maven plugins (or some of their commands aka goals) to a phase, some are bound to a phase by default.

Phase Description
validate Validate the project is correct and all necessary information is available.
initialize Initialize build state, e.g., set properties or create directories.
generate-sources Generate any source code for inclusion in compilation.
process-sources Process the source code, for example to filter any values.
generate-resources Generate resources for inclusion in the package.
process-resources Copy and process the resources into the destination directory, ready for packaging.
compile Compile the source code of the project.
process-classes Post-process the generated files from compilation, e.g., bytecode enhancement on Java classes.
generate-test-sources Generate any test source code for inclusion in compilation.
process-test-sources Process the test source code, for example to filter any values.
generate-test-resources Create resources for testing.
process-test-resources Copy and process the resources into the test destination directory.
test-compile Compile the test source code into the test destination directory.
process-test-classes Post-process the generated files from test compilation, e.g., bytecode enhancement on Java classes.
test Run tests using a suitable unit testing framework. These tests should not require packaging.
prepare-package Perform operations necessary to prepare a package before the actual packaging.
package Take the compiled code and package it in its distributable format, such as a JAR.
pre-integration-test Perform actions required before integration tests are executed, such as environment setup.
integration-test Process and deploy the package if necessary into an environment where integration tests can run.
post-integration-test Perform actions required after integration tests, including cleaning up the environment.
verify Run any checks to verify the package is valid and meets quality criteria.
install Install the package into the local repository for use as a dependency in other projects locally.
deploy Copies the final package to the remote repository for sharing with other developers and projects.

We for example bundled the webapps in generate-resources (which was used before by the frontend plugin we recently removed) as there was a race condition with packaging the dhis.war when I bound it to the package phase. This meant that we git cloned webapps when running tests. I found that the prepare-package phase works and means we do not waste time cloning if we just want to run our tests.

I changed our GitHub CI test jobs to only run up to test as we do not want to package the war in every unit and integration test job. The dhis.war is packaged in the run-api-tests.yml. You can see that I had to bind the coverage plugin to test as it runs in verify by default.

Maven Plugins

Jacoco plugin

We use the Jacoco Plugin to generate coverage reports for codecov.io.

jacoco:report-aggregate is bound to verify by default.

Move it earlier to test as shown by codecov

https://github.com/codecov/example-java-maven/blob/5b913c6d90c7a83520cf082c4053f5422cd1e9db/pom.xml#L39-L45

Maven Enforcer Plugin

We use the Maven enforcer to ensure we import the correct library if there are multiple choices with the same name, ...

We don't bind it to a different phase

dhis2-core/dhis-2/pom.xml

Lines 2254 to 2259 in fd4619f

<execution>
<id>enforce-maven</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>

so its default applies which is validate.

Maven Dependency Plugin

We use the Maven dependency plugin to ensure our builds are more reproducible. It forces us to declare used dependencies instead of relying on a dependency coming in transitively.

We did bind the plugin to the lifecycle phase process-test-classes so such problems show up early.

dhis2-core/dhis-2/pom.xml

Lines 2358 to 2363 in fd4619f

<execution>
<id>analyze</id>
<goals>
<goal>analyze-only</goal>
</goals>
<phase>process-test-classes</phase>

This annoyed devs as multiple test jobs would then fail for the same reason. So we now rely on its default binding to the lifecycle phase verify. Note that we therefore execute verify in the .github/workflows/run-api-tests.yml so this plugin gets executed when the war is built for the e2e tests.

Move it the bundling further down in the maven lifecycles
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html\#Lifecycle_Reference
Just before the package phase where the war is built
so it only runs once on GH and not during 'mvn test'. It will
run after the package phase.

Therefore change the run-api-tests to run 'mvn verify'. We run 'mvn
install' on Jenkins.
The maven dependency plugin thinks the deps are unused declared ones if we
run verify without compiling the test classes
Copy link

@teleivo teleivo added the deploy Deploy DHIS2 instance with IM. label Oct 22, 2024
@teleivo teleivo marked this pull request as ready for review October 22, 2024 07:20
@teleivo teleivo requested a review from a team as a code owner October 22, 2024 07:20
@teleivo teleivo merged commit af7757f into master Oct 22, 2024
18 checks passed
@teleivo teleivo deleted the build-webapps branch October 22, 2024 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deploy Deploy DHIS2 instance with IM.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants