forked from spring-cloud/spring-cloud-dataflow
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
The big whoppa
- Loading branch information
Showing
877 changed files
with
23,310 additions
and
24,989 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,8 +4,8 @@ if [ "$TAG" == "" ]; then | |
| exit 1 | ||
| fi | ||
| if [ "$DEFAULT_JDK" = "" ]; then | ||
| echo "DEFAULT_JDK not found using 11" | ||
| DEFAULT_JDK=11 | ||
| echo "DEFAULT_JDK not found using 17" | ||
| DEFAULT_JDK=17 | ||
| else | ||
| echo "DEFAULT_JDK=$DEFAULT_JDK" | ||
| fi | ||
|
|
@@ -20,7 +20,7 @@ function pack_image { | |
| fi | ||
| echo "Creating: $REPO:$TAG-jdk$v" | ||
| # --buildpack "paketo-buildpacks/[email protected]" --buildpack "paketo-buildpacks/[email protected]" | ||
| pack build --builder gcr.io/paketo-buildpacks/builder:base \ | ||
| pack build --builder paketobuildpacks/builder-jammy-base:latest \ | ||
| --path "$JAR" \ | ||
| --trust-builder --verbose \ | ||
| --env BP_JVM_VERSION=$v "$REPO:$TAG-jdk$v" | ||
|
|
@@ -37,7 +37,7 @@ for ((i = 0; i < LEN; i++)); do | |
| IMAGE="$(jq -r --argjson index $i '.include[$index] | .image' .github/workflows/images.json)" | ||
| ARTIFACT_ID="$(jq -r --argjson index $i '.include[$index] | .name' .github/workflows/images.json)" | ||
| # 8 11 17 21 | ||
| for v in 8 11 17; do | ||
| for v in 17 21; do | ||
| pack_image "$TARGET/$ARTIFACT_ID" $IMAGE $v $ARTIFACT_ID | ||
| RC=$? | ||
| if [ $RC -ne 0 ]; then | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,7 +28,7 @@ jobs: | |
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-java@v3 | ||
| with: | ||
| java-version: '8' | ||
| java-version: '17' | ||
| distribution: 'liberica' | ||
| - name: Load matrix | ||
| id: matrix | ||
|
|
@@ -60,7 +60,7 @@ jobs: | |
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-java@v3 | ||
| with: | ||
| java-version: '8' | ||
| java-version: '17' | ||
| distribution: 'liberica' | ||
| - name: Install pack | ||
| uses: jvalkeal/[email protected] | ||
|
|
@@ -81,7 +81,7 @@ jobs: | |
| shell: bash | ||
| env: | ||
| TAG: ${{ needs.prepare.outputs.version }} | ||
| DEFAULT_JDK: '11' | ||
| DEFAULT_JDK: '17' | ||
| ARTIFACTORY_USERNAME: ${{ secrets.ARTIFACTORY_USERNAME }} | ||
| ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }} | ||
| DELETE_TAGS: ${{ inputs.delete-tags }} | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,7 +3,7 @@ name: CI Carvel | |
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - main-3 | ||
| paths: | ||
| - 'src/carvel/**' | ||
| workflow_dispatch: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| = Workflow Reference | ||
|
|
||
| This README serves as a guide to the GitHub Action workflows included in this repository. | ||
| It outlines the purpose and functionality of each workflow, detailing their role in the CI and release processes. | ||
| Additionally, this document provides an overview of the scripts and actions utilized in these workflows, offering insights into how they work together in SCDF's CI/CD pipeline. | ||
|
|
||
| This document is a work in progress, and as various workflows are updated, the documentation will be revised to reflect both existing and new behaviors. | ||
|
|
||
|
|
||
| == Building Docker Images and pushing the containers to DockerHub | ||
| .This diagram shows the flow of execution of how workflows create Docker imges. | ||
| ``` | ||
| ┌─────────────────────────┐ | ||
| │ │ | ||
| │ │ | ||
| │build-snapshot-worker.yml┼────┐ | ||
| │ │ │ | ||
| │ │ │ | ||
| └─────────────────────────┘ │ | ||
| ┌─────────────────────────┐ │ | ||
| │ │ │ | ||
| │ │ │ | ||
| │ ci-images.yml ┼────┤ ┌─────────────────────────┐ ┌─────────────────────────┐ | ||
| │ │ │ │ │ │ │ | ||
| │ │ │ │ │ │ │ | ||
| └─────────────────────────┘ ├────►│ build-images.yml ┼────────►│ build-image.sh │ | ||
| ┌─────────────────────────┐ │ │ │ │ │ | ||
| │ │ │ │ │ │ │ | ||
| │ │ │ └───────────┬─────────────┘ └─────────────────────────┘ | ||
| │ ci.yml ┼────┤ │ | ||
| │ │ │ │ | ||
| │ │ │ ┌───────────┴─────────────┐ | ||
| └─────────────────────────┘ │ │ │ | ||
| ┌─────────────────────────┐ │ │ │ | ||
| │ │ │ │ images.json │ | ||
| │ │ │ │ │ | ||
| │ release-worker.yml ┼────┘ │ │ | ||
| │ │ └─────────────────────────┘ | ||
| │ │ | ||
| └─────────────────────────┘ | ||
| ``` | ||
|
|
||
| Part of the release and CI process involves creating and pushing images to a registry (such as DockerHub) for the Dataflow server, Skipper server, CTR app, and other components. | ||
| This process is managed by the `build-images` (build-images.yml) workflow. While the `build-images` workflow is typically not run independently, it is invoked by other workflows that handle CI builds and releases. | ||
| The `build-images` workflow determines which images to create based on the `images.json` file. | ||
| This file contains metadata on the primary SCDF components that need to have an associated image. | ||
| Each entry specifies the location (directory) where the jar can be found, jar name, and image name for each artifact that will be used to construct the image. | ||
| For each entry in the `images.json` file, the workflow calls the `build-image.sh` script, which retrieves the jar, builds the image, and then pushes it to the registry. | ||
|
|
||
| SCDF also provides images for external applications that support some of the optional features that are offered by dataflow. | ||
| These include Grafana and Prometheus local. | ||
| These images are created and pushed using the docker/build-push-action@v2 action. | ||
|
|
||
| === Scripts used to build images | ||
| As mentioned above, the `build-image.sh` script is responsible for building the specified image based on the parameters provided and then pushing the image to Dockerhub. | ||
| This script uses Paketo to build an image for each of the supported Java versions using the corresponding jar file. | ||
| The resulting image name will look something like `spring-cloud-dataflow-server:3.0.0-SNAPSHOT-jdk17`. | ||
| Additionally, the script creates a default image using the default Java version as specified by the `DEFAULT_JDK` environment variable. | ||
|
|
||
| The format for running the `build-image.sh` is as follows: | ||
| [source, bash] | ||
| ``` | ||
| bash | ||
| ./build-image.sh <directory containing the jar> <The name of the image to create> <name of the jar> | ||
| ``` | ||
|
|
||
| There is an optional `DEFAULT_JDK` environment variable that allows you to set the JDK version for the default image created. | ||
| If not the script will set it to its current setting (which as of the writing of this document is `17`). | ||
|
|
||
| NOTE: When new releases of Java are available and are compliant with the SCDF release, they need to be added to the `build-image.sh` script. | ||
|
|
Oops, something went wrong.