Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update build workflow to build indexer plugins #360

Merged
merged 28 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3446503
Update build workflow to include Wazuh plugins
AlexRuiz7 Aug 16, 2024
a06bed7
Try new approach to build wazuh-indexer with plugins
AlexRuiz7 Aug 16, 2024
20eddf0
Remove old code
AlexRuiz7 Aug 16, 2024
167809a
Remove ADMINS.md
AlexRuiz7 Aug 16, 2024
6ebc18f
Sync mavel local path across jobs
AlexRuiz7 Aug 20, 2024
1eb2403
Fix versioning of wazuh-indexer-plugins
AlexRuiz7 Aug 20, 2024
c693b68
Fix versioning of wazuh-indexer-plugins
AlexRuiz7 Aug 20, 2024
c444752
Pass version and revision to publishToMavenLocal
AlexRuiz7 Aug 20, 2024
7b74df4
Add version check test
AlexRuiz7 Aug 21, 2024
b4a61fa
Format files
AlexRuiz7 Aug 21, 2024
979407e
Use upload-artifact and download-artifact to share the plugins' zips …
AlexRuiz7 Aug 22, 2024
12eecba
Add repo path
AlexRuiz7 Aug 22, 2024
39d123f
Fix plugin name
AlexRuiz7 Aug 22, 2024
38ecf65
Roll back
AlexRuiz7 Aug 22, 2024
d4219f8
Remove exit 1
AlexRuiz7 Aug 22, 2024
4e5d0ea
Fix relative path to the plugins
AlexRuiz7 Aug 22, 2024
e415653
List plugins folder
AlexRuiz7 Aug 22, 2024
dd0560e
Fix relative path
AlexRuiz7 Aug 22, 2024
3feb43f
again
AlexRuiz7 Aug 22, 2024
758e572
Change relative path to absolute
AlexRuiz7 Aug 23, 2024
b9fa45b
Clean code
AlexRuiz7 Aug 23, 2024
7a3f180
Update README.ms
AlexRuiz7 Aug 23, 2024
667af71
Apply naming convention
AlexRuiz7 Aug 27, 2024
f74dd34
Add breif steps to build wazuh-indexer with plugins
AlexRuiz7 Aug 27, 2024
e9c3389
Skip job to build plugins on no input
AlexRuiz7 Aug 28, 2024
a031b89
Improve conditional
AlexRuiz7 Aug 28, 2024
ed86236
Remove build-plugins job from build's job dependencies
AlexRuiz7 Aug 28, 2024
d4c8bf6
Roll back
AlexRuiz7 Aug 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 75 additions & 18 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ on:
description: "ID used to identify the workflow uniquely."
type: string
required: false
plugins_reference:
description: "Branch, commit or tag for the plugins repository"
type: string
default: "master"
workflow_call:
inputs:
revision:
Expand Down Expand Up @@ -64,6 +68,10 @@ on:
id:
type: string
required: false
plugins_reference:
description: "Branch, commit or tag for the plugins repository"
type: string
default: "master"
secrets:
CI_INTERNAL_DEVELOPMENT_BUCKET_USER_ACCESS_KEY:
required: true
Expand Down Expand Up @@ -95,68 +103,117 @@ jobs:
- id: setup
run: |
matrix=$(jq -cn \
--argjson distribution '${{ inputs.distribution }}' \
--argjson architecture '${{ inputs.architecture }}' \
--argjson distribution '${{ github.event.inputs.distribution }}' \
--argjson architecture '${{ github.event.inputs.architecture }}' \
'{distribution: $distribution, architecture: $architecture}'
)
echo "matrix=$matrix" >> $GITHUB_OUTPUT

build-plugins:
if: ${{ github.event.inputs.plugins_reference != '' }}
strategy:
fail-fast: false
matrix:
plugins: ["wazuh-indexer-setup"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
repository: wazuh/wazuh-indexer-plugins
ref: ${{ github.event.inputs.plugins_reference }}
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 21

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4

- name: Get version
id: version
run: echo "version=$(<VERSION)" >> "$GITHUB_OUTPUT"

- name: Build with Gradle
working-directory: .//plugins/${{ matrix.plugins }}
run: |
./gradlew build -Dversion=${{ steps.version.outputs.version }} -Drevision=${{ github.event.inputs.revision }}
ls -lR build/distributions/

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.plugins }}
path: "./plugins/${{ matrix.plugins }}/build/distributions/${{ matrix.plugins }}-${{ steps.version.outputs.version }}.${{ github.event.inputs.revision }}.zip"
if-no-files-found: error

build:
needs: [matrix]
needs: [matrix, build-plugins]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.matrix.outputs.matrix) }}
steps:
- uses: actions/checkout@v4

# Download plugins
- uses: actions/download-artifact@v4
if: ${{ github.event.inputs.plugins_reference != '' }}
with:
path: ./artifacts/plugins
merge-multiple: true

- name: Display structure of downloaded files
if: ${{ github.event.inputs.plugins_reference != '' }}
run: ls -lR ./artifacts/plugins

- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 11
java-version: 21

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v3
uses: gradle/actions/setup-gradle@v4

- name: Provision
if: ${{ matrix.distribution == 'deb' }}
run: |
sudo bash packaging_scripts/provision.sh
sudo bash build-scripts/provision.sh

- name: Run `baptizer.sh` (min)
run: |
name=$(bash packaging_scripts/baptizer.sh -m \
name=$(bash build-scripts/baptizer.sh -m \
-a ${{ matrix.architecture }} \
-d ${{ matrix.distribution }} \
-r ${{ inputs.revision }} \
${{ inputs.is_stage && '-x' || '' }} \
-r ${{ github.event.inputs.revision }} \
${{ github.event.inputs.is_stage && '-x' || '' }} \
)
echo "name=$name" >> $GITHUB_OUTPUT
id: min_package

- name: Run `baptizer.sh`
run: |
name=$(bash packaging_scripts/baptizer.sh \
name=$(bash build-scripts/baptizer.sh \
-a ${{ matrix.architecture }} \
-d ${{ matrix.distribution }} \
-r ${{ inputs.revision }} \
${{ inputs.is_stage && '-x' || '' }} \
-r ${{ github.event.inputs.revision }} \
${{ github.event.inputs.is_stage && '-x' || '' }} \
)
echo "name=$name" >> $GITHUB_OUTPUT
id: package

- name: Run `build.sh`
run: |
bash packaging_scripts/build.sh \
bash build-scripts/build.sh \
-a ${{ matrix.architecture }} \
-d ${{ matrix.distribution }} \
-n ${{ steps.min_package.outputs.name }}

- name: Run `assemble.sh`
run: |
bash packaging_scripts/assemble.sh \
bash build-scripts/assemble.sh \
-a ${{ matrix.architecture }} \
-d ${{ matrix.distribution }} \
-r ${{ inputs.revision }}
-r ${{ github.event.inputs.revision }}

- name: Test RPM package
if: ${{ matrix.distribution == 'rpm' }}
Expand All @@ -180,15 +237,15 @@ jobs:
if-no-files-found: error

- name: Set up AWS CLI
if: ${{ inputs.upload }}
if: ${{ github.event.inputs.upload }}
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.CI_INTERNAL_DEVELOPMENT_BUCKET_USER_ACCESS_KEY }}
aws-secret-access-key: ${{ secrets.CI_INTERNAL_DEVELOPMENT_BUCKET_USER_SECRET_KEY }}
aws-region: ${{ secrets.CI_AWS_REGION }}

- name: Upload package to S3
if: ${{ inputs.upload }}
if: ${{ github.event.inputs.upload }}
run: |
src="artifacts/dist/${{ steps.package.outputs.name }}"
dest="s3://packages-dev.internal.wazuh.com/development/wazuh/4.x/main/packages/"
Expand All @@ -197,7 +254,7 @@ jobs:
echo "S3 URI: ${s3uri}"

- name: Upload checksum to S3
if: ${{ inputs.upload && inputs.checksum }}
if: ${{ github.event.inputs.upload && github.event.inputs.checksum }}
run: |
src="artifacts/dist/${{ steps.package.outputs.name }}.sha512"
dest="s3://packages-dev.internal.wazuh.com/development/wazuh/4.x/main/packages/"
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/version_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Version check

on:
push:
paths:
- "VERSION"

jobs:
check-version:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Check version
run: bash build-scripts/check-version.sh
Loading