Skip to content

Commit

Permalink
WIP Release infra stuff
Browse files Browse the repository at this point in the history 
Introduce a couple of shell scripts to automate the release process.
Some docs are included under `releases/README.md`.

Generally, releases at the ASF follow the following workflow:
1. Draft a release
2. Start a VOTE on the dev mailing list
3. If the VOTE fails, the release has failed - "go to step 1"
4. If the VOTE passes, publish the release

The above process is, without release branches, reflected in the scripts:
1. `releases/bin/draft-release.sh --major <major-version-number> --minor <minor-version-number> --commit <Git-commit-ID>`
2. if the vote passes: `releases/bin/publish-release.sh --major <major-version-number> --minor <minor-version-number>`
3. if the vote fails, just run `draft-release.sh` again

The change includes scripts to handle version branches, however, using those is not required for the two release scripts above.

It's important to know that the scripts handle changes to the `version.txt` file and that a specific syntax for Git tags is expected, which is required to automatically use/generate RC and final versions, including the artifact publishing via Sonatype.
  • Loading branch information
@snazy
snazy committed Dec 10, 2024
1 parent 426f5eb commit 25402f3
Show file tree
Hide file tree
Showing 15 changed files with 1,655 additions and 148 deletions.
18 changes: 18 additions & 0 deletions NEXT_RELEASE_NOTES.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
7 changes: 0 additions & 7 deletions build-logic/src/main/kotlin/publishing/PublishingHelperPlugin.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,8 @@ import org.gradle.plugins.signing.SigningPlugin
* The task `sourceTarball` (available on the root project) generates a source tarball using `git
* archive`.
*
* The task `releaseEmailTemplate` generates the release-vote email subject + body. Outputs on the
* console and in the `build/distributions/` directory.
*
* Signing tip: If you want to use `gpg-agent`, set the `useGpgAgent` Gradle project property
*
* The following command publishes the project artifacts to your local maven repository, generates
* the source tarball - and uses `gpg-agent` to sign all artifacts and the tarball. Note that this
* requires a Git tag!
*
* ```
* ./gradlew publishToMavenLocal sourceTarball -Prelease -PuseGpgAgent
* ```
Expand Down
141 changes: 12 additions & 129 deletions build-logic/src/main/kotlin/publishing/rootProject.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,11 @@

package publishing

import io.github.gradlenexus.publishplugin.NexusPublishExtension
import io.github.gradlenexus.publishplugin.NexusPublishPlugin
import io.github.gradlenexus.publishplugin.internal.StagingRepositoryDescriptorRegistryBuildService
import org.gradle.api.Project
import org.gradle.api.services.BuildServiceRegistration
import org.gradle.api.tasks.Delete
import org.gradle.api.tasks.Exec
import org.gradle.kotlin.dsl.apply
import org.gradle.kotlin.dsl.named
import org.gradle.kotlin.dsl.register
import org.gradle.plugins.signing.Sign

Expand All @@ -41,11 +38,22 @@ internal fun configureOnRootProject(project: Project) =
val isRelease = project.hasProperty("release")
val isSigning = isRelease || project.hasProperty("signArtifacts")

val cleanDistributionsDir = tasks.register<Delete>("cleanDistributionsDir")
cleanDistributionsDir.configure {
outputs.cacheIf { false }

val e = project.extensions.getByType(PublishingHelperExtension::class.java)
delete(e.distributionDir)
}

val sourceTarball = tasks.register<Exec>("sourceTarball")
sourceTarball.configure {
group = "build"
description =
"Generate a source tarball for a release to be uploaded to dist.apache.org/repos/dist"
outputs.cacheIf { false }

dependsOn(cleanDistributionsDir)

val e = project.extensions.getByType(PublishingHelperExtension::class.java)
doFirst { mkdir(e.distributionDir) }
Expand Down Expand Up @@ -84,129 +92,4 @@ internal fun configureOnRootProject(project: Project) =
}
sourceTarball.configure { finalizedBy(signSourceTarball) }
}

val releaseEmailTemplate = tasks.register("releaseEmailTemplate")
releaseEmailTemplate.configure {
group = "publishing"
description =
"Generate release-vote email subject + body, including the staging repository URL, if run during the Maven release."

mustRunAfter("initializeApacheStagingRepository")

doFirst {
val e = project.extensions.getByType(PublishingHelperExtension::class.java)
val asfName = e.asfProjectName.get()

val gitInfo = MemoizedGitInfo.gitInfo(rootProject)
val gitCommitId = gitInfo["Apache-Polaris-Build-Git-Head"]

val repos = project.extensions.getByType(NexusPublishExtension::class.java).repositories
val repo = repos.iterator().next()

val stagingRepositoryUrlRegistryRegistration =
gradle.sharedServices.registrations.named<
BuildServiceRegistration<StagingRepositoryDescriptorRegistryBuildService, *>
>(
"stagingRepositoryUrlRegistry"
)
val staginRepoUrl =
if (stagingRepositoryUrlRegistryRegistration.isPresent) {
val stagingRepositoryUrlRegistryBuildServiceRegistration =
stagingRepositoryUrlRegistryRegistration.get()
val stagingRepositoryUrlRegistryService =
stagingRepositoryUrlRegistryBuildServiceRegistration.getService()
if (stagingRepositoryUrlRegistryService.isPresent) {
val registry = stagingRepositoryUrlRegistryService.get().registry
try {
val stagingRepoDesc = registry.get(repo.name)
val stagingRepoId = stagingRepoDesc.stagingRepositoryId
"https://repository.apache.org/content/repositories/$stagingRepoId/"
} catch (e: IllegalStateException) {
"NO STAGING REPOSITORY ($e)"
}
} else {
"NO STAGING REPOSITORY (no registry service) !!"
}
} else {
"NO STAGING REPOSITORY (no build service) !!"
}

val asfProjectName = fetchAsfProjectName(asfName)

val versionNoRc = version.toString().replace("-rc-?[0-9]+".toRegex(), "")

val subjectFile = e.distributionFile("vote-email-subject.txt").relativeTo(projectDir)
val bodyFile = e.distributionFile("vote-email-body.txt").relativeTo(projectDir)

val emailSubject = "[VOTE] Release $asfProjectName $version"
subjectFile.writeText(emailSubject)

val emailBody =
"""
Hi everyone,
I propose that we release the following RC as the official
$asfProjectName $versionNoRc release.
* This corresponds to the tag: apache-$asfName-$version
* https://github.com/apache/$asfName/commits/apache-$asfName-$version
* https://github.com/apache/$asfName/tree/$gitCommitId
The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/incubator/$asfName/apache-$asfName-$version
You can find the KEYS file here:
* https://dist.apache.org/repos/dist/release/incubator/$asfName/KEYS
Convenience binary artifacts are staged on Nexus. The Maven repository URL is:
* $staginRepoUrl
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache $asfName $version
[ ] +0
[ ] -1 Do not release this because...
Only PPMC members and mentors have binding votes, but other community members are
encouraged to cast non-binding votes. This vote will pass if there are
3 binding +1 votes and more binding +1 votes than -1 votes.
NB: if this vote pass, a new vote has to be started on the Incubator general mailing
list.
Thanks
Regards
"""

logger.lifecycle(
"""
The email for your release vote mail:
-------------------------------------
Suggested subject: (also in file $subjectFile)
$emailSubject
Suggested body: (also in file $bodyFile)
$emailBody
"""
.trimIndent()
)
bodyFile.writeText(emailBody.trimIndent())
}
}

if (isRelease) {
sourceTarball.configure { finalizedBy(releaseEmailTemplate) }
}

afterEvaluate {
tasks.named("closeApacheStagingRepository") { mustRunAfter(releaseEmailTemplate) }
}
}
103 changes: 103 additions & 0 deletions releases/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Apache Polaris Release Infrastructure

This directory holds all the infrastructure parts that are required to draft a Polaris release (RC) and to eventually
release it.

Releases can be created from any Git commit, version branches are supported.

Generally, releases at the ASF follow the following workflow:

1. Draft a release
2. Start a VOTE on the dev mailing list
3. If the VOTE fails, the release has failed - "go to step 1"
4. If the VOTE passes, publish the release

The Polaris project decided to run all release related via GitHub actions - a release is never drafted nor actually
released from a developer machine.

## Technical process

In the `release/bin/` directory are the scripts that are required to draft a release and to publish it as "GA". All
scripts can be called with the `--help` argument to get some usage information. There is also a `--dry-run` option
to inspect what _would_ happen.

The technical process for a release follows the workflow above:

1. `releases/bin/draft-release.sh --major <major-version-number> --minor <minor-version-number> --commit <Git-commit-ID>`
creates a release-candidate. The `--commit` argument is optional and defaults to the HEAD of the local Git
worktree. The patch version number and RC-number are generated automatically. This means, that RC-numbers are
automatically incremented as long as there is no "final" release tag. In that case, the patch version number
is incremented and the RC-number is set to 1.

The Git tag name that will be used follows the pattern `polaris-<major>.<minor>.<patch>-RC<rc-number>`

The content of the `version.txt` file is set to the full release version, for example `1.2.3`.

Gradle will run with the arguments `-Prelease publishToApache closeApacheStagingRepository sourceTarball`.
Both the release artifacts (jars) and the source tarball are signed. The signing and Nexus credentials must be
provided externally using the `ORG_GRADLE_PROJECT_*` environment variables. In dry-run mode, this step runs
Gradle with the arguments `-PjarWithGitInfo -PsignArtifacts -PuseGpgAgent publishToMavenLocal sourceTarball publishToMavenLocal`.

The staging repository ID, which is needed to release the staging repository, and URL are extracted from the
Gradle output and memoized in the files `releases/current-release-staging-repository-id` and
`releases/current-release-staging-repository-url`. The source Git commit ID is memoized in the file
`releases/current-release-commit-id`.

The source tarball will be uploaded to the Apache infrastructure.

Release notes will be generated and included in the file `site/content/in-dev/unreleasd/release-notes.md`. This
makes the release notes available later on the project website within the versioned docs.

The suggested release VOTE email subject and body with the correct links and information are provided.

The last step is to push the Git tag with a Git commit containing the above file changes.
2. Once the release VOTE passed: `releases/bin/publish-release.sh --major <major-version-number> --minor <minor-version-number>`.

The command will find the latest RC tag for the latest patch release of the given major/minor version.

The final Git tag name that will be used follows the pattern `polaris-<major>.<minor>.<patch>` and created from
the latest RC-tag for that version.

The Git tag is then pushed.

Documentation pages for the release will then be copied from the `site/content/in-dev/unreleased` into the
`site/content/releases/<major>.<minor>.<patch>` folder within the `versioned-docs` branch.
The changes for the updated `versioned-docs` branch are then pushed to Git.

The suggested ANNOUNCEMENT email subject and body are provided.

Creating a release in GitHub is the last step.

## Technical requirements

To use the scripts in the `releases/bin/` folder, it is required to have a _full_ clone with all tags and branches.
Shallow clones, which is the default when checking out a Git repository in GitHub actions, will _not_ work properly!

On top, all scripts need privileges to be able to push tags and branches to the GitHub repository - this is an
essential requirement for the scripts to do their job.

The `draft-release.sh` and `publish-release.sh` scripts also require the following environment variables providing
the necessary secrets:

* `ORG_GRADLE_PROJECT_signingKey`
* `ORG_GRADLE_PROJECT_signingPassword`
* `ORG_GRADLE_PROJECT_sonatypeUsername`
* `ORG_GRADLE_PROJECT_sonatypePassword`

GitHub actions running the scripts must provide those secrets and privileges.

## Version branches

The Polaris project may use maintenance version branches following the pattern `release/<major>.x` and
`release/<major>.<minor>`. The scripts mentioned above are already support version branches and have validations for
this use case. Using version branches is not a requirement for the release scripts.

The two scripts `releases/bin/create-release-branch.sh` plus the informative `releases/bin/list-release-branches.sh`
are there to help with version branches. The former must be invoked on the main branch and creates a new major-version
branch using the `release/<major>.x` pattern. The latter must be invoked on a major-version branch and creates a new
minor version branch using the `release/<major>.<minor>` pattern.

# TODO

* Close staging repo when RC is abandoned
* Adopt website (top bar menu, maintained releases, left site menu)
Loading

0 comments on commit 25402f3

Please sign in to comment.