Merge branch 'multiarch-push' into multiarch-push-robuust

# Conflicts: # .github/workflows/ci.yml # heroku-22-build/setup.sh
robuust · Apr 8, 2024 · 3444098 · 3444098
2 parents 6a4861c + 13c3432
commit 3444098
Show file tree

Hide file tree

Showing 40 changed files with 1,114 additions and 1,209 deletions.
diff --git a/BUILD.md b/BUILD.md
@@ -46,19 +46,14 @@ of 4 images:
 We use GitHub Actions to build and release Heroku Base Images:
 
 * Any push to `main` will build the images and push the nightly Docker tag variants (such as `heroku/heroku:22-build.nightly`).
-* Any new Git tag will build the image and push the latest Docker tag (such as `heroku/heroku:22-build`), as well as a versioned tag (such as `heroku/heroku:22-build.v89`).
+* Any new Git tag will build the image and push the latest Docker tag (such as `heroku/heroku:22-build`),
+  as well as a versioned tag (such as `heroku/heroku:22-build.v123`). The Docker image will then also be
+  converted to a Heroku-specific `.img` format and uploaded to S3 for consumption by the runtime hosts.
 
-# Releasing Heroku Base Images Locally (Prime)
+# Generating `.img` format Base Images locally
 
-When building Heroku Base Images for release locally, you'll need a number of additional steps.
+To test the generation of the Heroku-specific `.img` file:
 
-NOTE: These steps do *not* apply to `*cnb*` images.
-
-    export DOCKER_DEFAULT_PLATFORM=linux/amd64
-    # Build the base image(s) as you would above
-    # …
-    docker build ./tools -t heroku/image-tools
-    # SET MANIFEST_APP_URL and MANIFEST_APP_TOKEN values, this is the app that controls the bucket for images and metadata about the images (Cheverny)
-    docker run -it --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -e "MANIFEST_APP_URL=$MANIFEST_APP_URL" -e "MANIFEST_APP_TOKEN=$MANIFEST_APP_TOKEN" heroku/image-tools STACK
-    # this will use your local docker image and convert it to a heroku base image
-    # it will then upload this image and the staging manifest via the MANIFEST_APP
+1. Build the Docker images for your chosen stack as normal above.
+2. `docker build --platform=linux/amd64 ./tools -t heroku-image-tools`
+3. `docker run -it --rm --platform=linux/amd64 --privileged -v /var/run/docker.sock:/var/run/docker.sock heroku-image-tools STACK` (where `STACK` is the full stack name like `heroku-22`)
diff --git a/README.md b/README.md
@@ -5,20 +5,20 @@
 This repository holds recipes for building the base images for [Heroku stacks](https://devcenter.heroku.com/articles/stack).
 The recipes are also rendered into Docker images that are available on Docker Hub:
 
-| Image                                     | Base                                  | Type                    | Status      |
-|-------------------------------------------|---------------------------------------|-------------------------|-------------|
-| [heroku/heroku:18][heroku-tags]           | [ubuntu:18.04][ubuntu-tags]           | Heroku Base Run Image   | End-of-life |
-| [heroku/heroku:18-build][heroku-tags]     | [heroku/heroku:18][heroku-tags]       | Heroku Base Build Image | End-of-life |
-| [heroku/heroku:18-cnb][heroku-tags]       | [heroku/heroku:18][heroku-tags]       | CNB Base Run Image      | End-of-life |
-| [heroku/heroku:18-cnb-build][heroku-tags] | [heroku/heroku:18-build][heroku-tags] | CNB Base Build Image    | End-of-life |
-| [heroku/heroku:20][heroku-tags]           | [ubuntu:20.04][ubuntu-tags]           | Heroku Base Run Image   | Available   |
-| [heroku/heroku:20-build][heroku-tags]     | [heroku/heroku:20][heroku-tags]       | Heroku Base Build Image | Available   |
-| [heroku/heroku:20-cnb][heroku-tags]       | [heroku/heroku:20][heroku-tags]       | CNB Base Run Image      | Available   |
-| [heroku/heroku:20-cnb-build][heroku-tags] | [heroku/heroku:20-build][heroku-tags] | CNB Base Build Image    | Available   |
-| [heroku/heroku:22][heroku-tags]           | [ubuntu:22.04][ubuntu-tags]           | Heroku Base Run Image   | Recommended |
-| [heroku/heroku:22-build][heroku-tags]     | [heroku/heroku:22][heroku-tags]       | Heroku Base Build Image | Recommended |
-| [heroku/heroku:22-cnb][heroku-tags]       | [heroku/heroku:22][heroku-tags]       | CNB Base Run Image      | Recommended |
-| [heroku/heroku:22-cnb-build][heroku-tags] | [heroku/heroku:22-build][heroku-tags] | CNB Base Build Image    | Recommended |
+| Image                                     | Base                                  | Type               | Status      |
+|-------------------------------------------|---------------------------------------|--------------------|-------------|
+| [heroku/heroku:18][heroku-tags]           | [ubuntu:18.04][ubuntu-tags]           | Heroku Run Image   | End-of-life |
+| [heroku/heroku:18-build][heroku-tags]     | [heroku/heroku:18][heroku-tags]       | Heroku Build Image | End-of-life |
+| [heroku/heroku:18-cnb][heroku-tags]       | [heroku/heroku:18][heroku-tags]       | CNB Run Image      | End-of-life |
+| [heroku/heroku:18-cnb-build][heroku-tags] | [heroku/heroku:18-build][heroku-tags] | CNB Build Image    | End-of-life |
+| [heroku/heroku:20][heroku-tags]           | [ubuntu:20.04][ubuntu-tags]           | Heroku Run Image   | Available   |
+| [heroku/heroku:20-build][heroku-tags]     | [heroku/heroku:20][heroku-tags]       | Heroku Build Image | Available   |
+| [heroku/heroku:20-cnb][heroku-tags]       | [heroku/heroku:20][heroku-tags]       | CNB Run Image      | Available   |
+| [heroku/heroku:20-cnb-build][heroku-tags] | [heroku/heroku:20-build][heroku-tags] | CNB Build Image    | Available   |
+| [heroku/heroku:22][heroku-tags]           | [ubuntu:22.04][ubuntu-tags]           | Heroku Run Image   | Recommended |
+| [heroku/heroku:22-build][heroku-tags]     | [heroku/heroku:22][heroku-tags]       | Heroku Build Image | Recommended |
+| [heroku/heroku:22-cnb][heroku-tags]       | [heroku/heroku:22][heroku-tags]       | CNB Run Image      | Recommended |
+| [heroku/heroku:22-cnb-build][heroku-tags] | [heroku/heroku:22-build][heroku-tags] | CNB Build Image    | Recommended |
 
 ### Learn more
 

diff --git a/heroku-20-build/Dockerfile b/heroku-20-build/Dockerfile
@@ -1,4 +1,3 @@
 ARG BASE_IMAGE=heroku/heroku:20
 FROM $BASE_IMAGE
-COPY setup.sh /tmp/setup.sh
-RUN /tmp/setup.sh
+RUN --mount=target=/build /build/setup.sh
diff --git a/heroku-20-build/installed-packages.txt b/heroku-20-build/installed-packages.txt
@@ -561,10 +561,10 @@ pinentry-curses
 pkg-config
 poppler-data
 poppler-utils
-postgresql-client-15
+postgresql-client-16
 postgresql-client-common
 postgresql-common
-postgresql-server-dev-15
+postgresql-server-dev-16
 procps
 python-is-python3
 python2

diff --git a/heroku-20-build/setup.sh b/heroku-20-build/setup.sh
@@ -1,92 +1,91 @@
 #!/usr/bin/env bash
 
-set -euo pipefail
-
-# Redirect stderr to stdout since tracing/apt-get/dpkg spam it for things that aren't errors.
-exec 2>&1
-set -x
+set -euxo pipefail
 
 export DEBIAN_FRONTEND=noninteractive
 
-apt-get update
-apt-get install -y --no-install-recommends \
-    autoconf \
-    automake \
-    bison \
-    build-essential \
-    bzr \
-    cmake \
-    gettext \
-    git \
-    jq \
-    libacl1-dev \
-    libapt-pkg-dev \
-    libargon2-dev \
-    libattr1-dev \
-    libaudit-dev \
-    libbsd-dev \
-    libbz2-dev \
-    libc-client2007e-dev \
-    libcairo2-dev \
-    libcap-dev \
-    libcurl4-openssl-dev \
-    libdb-dev \
-    libev-dev \
-    libevent-dev \
-    libexif-dev \
-    libffi-dev \
-    libgcrypt20-dev \
-    libgd-dev \
-    libgdbm-dev \
-    libgeoip-dev \
-    libglib2.0-dev \
-    libgnutls28-dev \
-    libgs-dev \
-    libicu-dev \
-    libidn11-dev \
-    libjpeg-dev \
-    libkeyutils-dev \
-    libkmod-dev \
-    libkrb5-dev \
-    libldap2-dev \
-    liblz4-dev \
-    liblzf-dev \
-    libmagic-dev \
-    libmagickwand-dev \
-    libmcrypt-dev \
-    libmemcached-dev \
-    libmysqlclient-dev \
-    libncurses5-dev \
-    libncursesw5-dev \
-    libnetpbm10-dev \
-    libonig-dev \
-    libpam0g-dev \
-    libpopt-dev \
-    libpq-dev \
-    librabbitmq-dev \
-    libreadline-dev \
-    librtmp-dev \
-    libseccomp-dev \
-    libselinux1-dev \
-    libsemanage1-dev \
-    libsodium-dev \
-    libssl-dev \
-    libsystemd-dev \
-    libtool \
-    libudev-dev \
-    libuv1-dev \
-    libwrap0-dev \
-    libxml2-dev \
-    libxslt-dev \
-    libyaml-dev \
-    libzip-dev \
-    libzstd-dev \
-    mercurial \
-    patchelf \
-    postgresql-server-dev-15 \
-    python3-dev \
-    ruby-dev \
-    zlib1g-dev \
+packages=(
+  autoconf
+  automake
+  bison
+  build-essential
+  bzr
+  cmake
+  gettext
+  git
+  jq
+  libacl1-dev
+  libapt-pkg-dev
+  libargon2-dev
+  libattr1-dev
+  libaudit-dev
+  libbsd-dev
+  libbz2-dev
+  libc-client2007e-dev
+  libcairo2-dev
+  libcap-dev
+  libcurl4-openssl-dev
+  libdb-dev
+  libev-dev
+  libevent-dev
+  libexif-dev
+  libffi-dev
+  libgcrypt20-dev
+  libgd-dev
+  libgdbm-dev
+  libgeoip-dev
+  libglib2.0-dev
+  libgnutls28-dev
+  libgs-dev
+  libicu-dev
+  libidn11-dev
+  libjpeg-dev
+  libkeyutils-dev
+  libkmod-dev
+  libkrb5-dev
+  libldap2-dev
+  liblz4-dev
+  liblzf-dev
+  libmagic-dev
+  libmagickwand-dev
+  libmcrypt-dev
+  libmemcached-dev
+  libmysqlclient-dev
+  libncurses5-dev
+  libncursesw5-dev
+  libnetpbm10-dev
+  libonig-dev
+  libpam0g-dev
+  libpopt-dev
+  libpq-dev
+  librabbitmq-dev
+  libreadline-dev
+  librtmp-dev
+  libseccomp-dev
+  libselinux1-dev
+  libsemanage1-dev
+  libsodium-dev
+  libssl-dev
+  libsystemd-dev
+  libtool
+  libudev-dev
+  libuv1-dev
+  libwrap0-dev
+  libxml2-dev
+  libxslt-dev
+  libyaml-dev
+  libzip-dev
+  libzstd-dev
+  mercurial
+  patchelf
+  postgresql-server-dev-16
+  python3-dev
+  ruby-dev
+  zlib1g-dev
+)
+
+apt-get update --error-on=any
+apt-get install -y --no-install-recommends "${packages[@]}"
 
 rm -rf /root/*
 rm -rf /tmp/*

diff --git a/heroku-20-cnb-build/Dockerfile b/heroku-20-cnb-build/Dockerfile
@@ -1,18 +1,23 @@
 ARG BASE_IMAGE=heroku/heroku:20-build
 FROM $BASE_IMAGE
 
-RUN groupadd heroku --gid 1000 && \
-  useradd heroku -u 1000 -g 1000 -s /bin/bash -m
-
-RUN mkdir /app && \
-  chown heroku:heroku /app
+RUN groupadd heroku --gid 1000 \
+  && useradd heroku --uid 1000 --gid 1000 --shell /bin/bash --create-home \
+  && mkdir /app \
+  && chown heroku:heroku /app
 
+# https://github.com/buildpacks/spec/blob/platform/0.13/platform.md#build-image
+USER heroku
 ENV CNB_USER_ID=1000
 ENV CNB_GROUP_ID=1000
-ENV CNB_STACK_ID "heroku-20"
-ENV CNB_TARGET_OS="linux"
-ENV CNB_TARGET_ARCH="amd64"
+# Note: This image doesn't inherit from the CNB run image variant so we have
+# to redeclare the labels present in the CNB run image again here.
+LABEL io.buildpacks.base.distro.name="ubuntu"
+LABEL io.buildpacks.base.distro.version="20.04"
+LABEL io.buildpacks.base.homepage="https://github.com/heroku/base-images"
+LABEL io.buildpacks.base.maintainer="Heroku"
 
+# Stack IDs are deprecated, but we still set these for backwards compatibility:
+# https://github.com/buildpacks/spec/blob/platform/0.13/platform.md#iobuildpacksstack-labels
+ENV CNB_STACK_ID="heroku-20"
 LABEL io.buildpacks.stack.id="heroku-20"
-
-USER heroku
diff --git a/heroku-20-cnb/Dockerfile b/heroku-20-cnb/Dockerfile
@@ -1,11 +1,19 @@
 ARG BASE_IMAGE=heroku/heroku:20
 FROM $BASE_IMAGE
 
-RUN ln -s /workspace /app
+RUN groupadd heroku --gid 1000 \
+  && useradd heroku --uid 1000 --gid 1000 --shell /bin/bash --create-home \
+  && ln -s /workspace /app
 
-RUN groupadd heroku --gid 1000 && \
-  useradd heroku -u 1000 -g 1000 -s /bin/bash -m
+# https://github.com/buildpacks/spec/blob/platform/0.13/platform.md#run-image
+USER heroku
+LABEL io.buildpacks.base.distro.name="ubuntu"
+LABEL io.buildpacks.base.distro.version="20.04"
+LABEL io.buildpacks.base.homepage="https://github.com/heroku/base-images"
+LABEL io.buildpacks.base.maintainer="Heroku"
 
+# Stack IDs are deprecated, but we still set this for backwards compatibility:
+# https://github.com/buildpacks/spec/blob/platform/0.13/platform.md#iobuildpacksstack-labels
 LABEL io.buildpacks.stack.id="heroku-20"
-USER heroku
-ENV HOME /app
+
+ENV HOME=/app
diff --git a/heroku-20/Dockerfile b/heroku-20/Dockerfile
@@ -1,4 +1,2 @@
 FROM ubuntu:20.04
-ARG TARGETARCH
-COPY setup.sh /tmp/setup.sh
-RUN /tmp/setup.sh
+RUN --mount=target=/build /build/setup.sh
diff --git a/heroku-20/imagemagick-policy.xml b/heroku-20/imagemagick-policy.xml
@@ -0,0 +1,13 @@
+<policymap>
+  <policy domain="resource" name="memory" value="256MiB"/>
+  <policy domain="resource" name="map" value="512MiB"/>
+  <policy domain="resource" name="width" value="16KP"/>
+  <policy domain="resource" name="height" value="16KP"/>
+  <policy domain="resource" name="area" value="128MB"/>
+  <policy domain="resource" name="disk" value="1GiB"/>
+  <policy domain="delegate" rights="none" pattern="URL" />
+  <policy domain="delegate" rights="none" pattern="HTTPS" />
+  <policy domain="delegate" rights="none" pattern="HTTP" />
+  <policy domain="path" rights="none" pattern="@*"/>
+  <policy domain="cache" name="shared-secret" value="passphrase" stealth="true"/>
+</policymap>
diff --git a/heroku-20/installed-packages.txt b/heroku-20/installed-packages.txt
@@ -355,7 +355,7 @@ perl-modules-5.30
 pinentry-curses
 poppler-data
 poppler-utils
-postgresql-client-15
+postgresql-client-16
 postgresql-client-common
 procps
 python-is-python3