Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: build multi-platform image with cross-compilation #12888

Closed
wants to merge 2 commits into from

Conversation

dvdksn
Copy link
Contributor

@dvdksn dvdksn commented Sep 29, 2024

The previous build workflow used emulation to build the Docker image,
which results in a somewhat complicated push-by-digest and merge
workflow to create a multi-platform image.

This commit changes the Docker build to use cross-compilation instead,
resulting in a faster and more straightforward build.

Note

Being not so familiar with Mage, I decided to use plain go build in the Dockerfile. I encountered some issues with Mage where the compiled binary couldn't be found for some reason. I'm sure there is a way to do it with Mage but I didn't want to spend the time debugging it.

Signed-off-by: David Karlsson [email protected]

@dvdksn dvdksn marked this pull request as ready for review September 29, 2024 17:40
@dvdksn
Copy link
Contributor Author

dvdksn commented Sep 29, 2024

cc @crazy-max if you want to take a look 🙏🏻

@bep
Copy link
Member

bep commented Sep 30, 2024

Thanks for this,

I guess this also enables the extended build (optional, default off), which is fined by me, just wanted to make a note of it. I guess it will make the Docker image slightly bigger, which will annoy some people, but I think people will expect the extended version.

@dvdksn I will test the Docker file myself, but have you tested the GitHub actions workflow? (I suspect we need to somehow trigger test builds of this, but I'm a little bit tired of that story for now).

@jmooring if we leave the parts about asciidoctor, pandoc etc. out of this for now, do you agree with this new Dockerfile? In my head, the default value for HUGO_BUILD_TAGS should be extended?

@dvdksn
Copy link
Contributor Author

dvdksn commented Sep 30, 2024

I guess this also enables the extended build (optional, default off)

I don't think it does at the moment; HUGO_BUILD_TAGS defaults to none also with this build.

    xx-go build -tags "${HUGO_BUILD_TAGS:-none}" -o /usr/bin/hugo

I haven't tested the GitHub Actions workflow, but I'll run through it on a fork to verify that it works. Btw, this will also result push a latest tag of the image for the highest tag version (determined automatically by docker/metadata-action)

If you want, I could set up the workflow so that we push both the extended and regular variant of the image, under different tags. E.g.

  • ghcr.io/gohugoio/hugo:v0.135.0
  • ghcr.io/gohugoio/hugo:v0.135.0-extended

@jmooring
Copy link
Member

I agree that the extended build is an incremental improvement, but I wouldn't spend any more time on this unless it gets some traction.

@bep
Copy link
Member

bep commented Sep 30, 2024

If you want, I could set up the workflow so that we push both the extended and regular variant of the image, under different tags. E.g.

No, I don't want to have several versions of the image to confuse users further. I will think about what to set as the sensible default, but I will adjust the Dockerfile once I make up my mind.

Btw, this will also result push a latest tag of the image for the highest tag version (determined automatically by docker/metadata-action)

That's fine.

The previous build workflow used emulation to build the Docker image,
which results in a somewhat complicated push-by-digest and merge
workflow to create a multi-platform image.

This commit changes the Docker build to use cross-compilation instead,
resulting in a faster and more straightforward build.

Signed-off-by: David Karlsson <[email protected]>
@dvdksn
Copy link
Contributor Author

dvdksn commented Sep 30, 2024

@bep here's a dry-run of the gha workflow on my fork: https://github.com/dvdksn/hugo/actions/runs/11102111454/job/30841202502

the image is published as ghcr.io/dvdksn/hugo:latest
https://explore.ggcr.dev/?image=ghcr.io%2Fdvdksn%2Fhugo%3Alatest

id: build
uses: docker/build-push-action@16ebe778df0e7752d2cfcbd924afdbbd89c1a755 # v6.6.1
with:
context: .
provenance: mode=max
sbom: true
push: ${{ startsWith(github.ref, 'refs/tags') }}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this workflow is triggered only on release event I think push: true is enough but doesn't hurt to check ref.

We should also build on PR to make sure the workflow runs fine imo:

on:
  release:
    types: [published]
  pull_request:
permissions:
  packages: write

Then push can still be:

push: ${{ startsWith(github.ref, 'refs/tags') }}

Or

push: ${{ github.event_name != 'pull_request' }}

Can be a follow-up if you want

@crazy-max
Copy link

crazy-max commented Sep 30, 2024

If you want, I could set up the workflow so that we push both the extended and regular variant of the image, under different tags. E.g.

  • ghcr.io/gohugoio/hugo:v0.135.0
  • ghcr.io/gohugoio/hugo:v0.135.0-extended

I think variants would make sense, this is also a common thing for official images like https://hub.docker.com/_/debian:

  • bookworm
  • bookworm-slim

It could just be a matrix setting HUGO_BUILD_TAGS build arg and adjust the metadata-action to set the suffix variant:

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        tag:
          - none
          - extended
    steps:
      - name: Prepare
        run: |
          tag="${{ matrix.tag }}"
          if [ "$tag" != "none" ]; then
            echo "IMAGE_SUFFIX=-${tag}" >> $GITHUB_ENV
          fi

      - name: Checkout
        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1
        with:
          images: ${{ env.REGISTRY_IMAGE }}
          flavor: |
            suffix=${{ env.IMAGE_SUFFIX }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1

      - name: Login to GHCR
        # Login is only needed when the image is pushed
        uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        id: build
        uses: docker/build-push-action@16ebe778df0e7752d2cfcbd924afdbbd89c1a755 # v6.6.1
        with:
          context: .
          provenance: mode=max
          sbom: true
          build-args: |
            HUGO_BUILD_TAGS=${{ matrix.tag }}
          push: ${{ github.event_name != 'pull_request' }}
          platforms: linux/amd64,linux/arm64
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

@bep
Copy link
Member

bep commented Oct 9, 2024

Thanks for this, much appreciated. I some additional things I want to append to this (Go version upgrade) and I will test this and complete this in #12921.

@bep bep closed this Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants