diff --git a/.gitignore b/.gitignore
index 4a7d8b9..64c67bf 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,7 +6,6 @@ SKIP
SKIP_IMAGES
.pc
*-pc
-apt-cacher-ng/
#
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000..32f2688
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,4 @@
+include:
+ - project: serge/pi-gen
+ ref: ci
+ file: 'pi-gen.yml'
diff --git a/Dockerfile b/Dockerfile
index 2a5d8fe..0e6d86d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -7,8 +7,8 @@ RUN apt-get -y update && \
apt-get -y install --no-install-recommends \
git vim parted \
quilt coreutils qemu-user-static debootstrap zerofree zip dosfstools \
- libarchive-tools libcap2-bin rsync grep udev xz-utils curl xxd file kmod bc\
- binfmt-support ca-certificates qemu-utils kpartx fdisk gpg pigz\
+ libarchive-tools libcap2-bin rsync grep udev xz-utils curl xxd file kmod bc \
+ binfmt-support ca-certificates fdisk gpg pigz arch-test \
&& rm -rf /var/lib/apt/lists/*
COPY . /pi-gen/
diff --git a/GL-README.md b/GL-README.md
index b38ec80..0843d87 100644
--- a/GL-README.md
+++ b/GL-README.md
@@ -13,7 +13,7 @@ There are several different images available, depending on your needs, from smal
## Source Code
-This build system is based on [pi-gen](https://github.com/RPi-Distro/pi-gen). Refer to its [original README](/README.md) for how everything works. The (`gl-config`)[gl-config] file is the key source of control. (What is called "config" in the original.)
+This build system is based on [pi-gen](https://github.com/RPi-Distro/pi-gen). Refer to its [original README](/README.md) for how everything works. The [`gl-config`](gl-config) file is the key source of control. (What is called "config" in the original.)
Note that we're tracking the `arm64` branch, not main. (If we build off the main branch, we hit [an issue with missing `arm/v8` docker images](https://github.com/groundlight/monitoring-notification-server/issues/39) and likely others, because we make these funky machines with a 64-bit kernel, but 32-bit applications.)
diff --git a/README.md b/README.md
index 7416931..439c690 100644
--- a/README.md
+++ b/README.md
@@ -1,21 +1,25 @@
# pi-gen
-Tool used to create Raspberry Pi OS images. (Previously known as Raspbian).
+Tool used to create Raspberry Pi OS images, and custom images based on Raspberry Pi OS,
+which was in turn derived from the Raspbian project.
+**Note**: Raspberry Pi OS 32 bit images are based primarily on Raspbian, while
+Raspberry Pi OS 64 bit images are based primarily on Debian.
## Dependencies
-pi-gen runs on Debian-based operating systems. Currently it is only supported on
-either Debian Buster or Ubuntu Xenial and is known to have issues building on
-earlier releases of these systems. On other Linux distributions it may be possible
-to use the Docker build described below.
+pi-gen runs on Debian-based operating systems released after 2017, and we
+always advise you use the latest OS for security reasons.
+
+On other Linux distributions it may be possible to use the Docker build described
+below.
To install the required dependencies for `pi-gen` you should run:
```bash
apt-get install coreutils quilt parted qemu-user-static debootstrap zerofree zip \
dosfstools libarchive-tools libcap2-bin grep rsync xz-utils file git curl bc \
-qemu-utils kpartx gpg pigz
+gpg pigz xxd arch-test
```
The file `depends` contains a list of tools needed. The format of this
@@ -48,48 +52,24 @@ environment variables.
The following environment variables are supported:
- * `IMG_NAME` **required** (Default: unset)
+ * `IMG_NAME` (Default: `raspios-$RELEASE-$ARCH`, for example: `raspios-bookworm-armhf`)
- The name of the image to build with the current stage directories. Setting
- `IMG_NAME=Raspbian` is logical for an unmodified RPi-Distro/pi-gen build,
- but you should use something else for a customized version. Export files
- in stages may add suffixes to `IMG_NAME`.
+ The name of the image to build with the current stage directories. Use this
+ variable to set the root name of your OS, eg `IMG_NAME=Frobulator`.
+ Export files in stages may add suffixes to `IMG_NAME`.
* `PI_GEN_RELEASE` (Default: `Raspberry Pi reference`)
The release name to use in `/etc/issue.txt`. The default should only be used
for official Raspberry Pi builds.
-* `USE_QCOW2` **EXPERIMENTAL** (Default: `0` )
-
- Instead of using traditional way of building the rootfs of every stage in
- single subdirectories and copying over the previous one to the next one,
- qcow2 based virtual disks with backing images are used in every stage.
- This speeds up the build process and reduces overall space consumption
- significantly.
-
- Additional optional parameters regarding qcow2 build:
-
- * `BASE_QCOW2_SIZE` (Default: 12G)
-
- Size of the virtual qcow2 disk.
- Note: it will not actually use that much of space at once but defines the
- maximum size of the virtual disk. If you change the build process by adding
- a lot of bigger packages or additional build stages, it can be necessary to
- increase the value because the virtual disk can run out of space like a normal
- hard drive would.
-
- **CAUTION:** Although the qcow2 build mechanism will run fine inside Docker, it can happen
- that the network block device is not disconnected correctly after the Docker process has
- ended abnormally. In that case see [Disconnect an image if something went wrong](#Disconnect-an-image-if-something-went-wrong)
-
-* `RELEASE` (Default: bookworm)
+* `RELEASE` (Default: `bookworm`)
The release version to build images against. Valid values are any supported
Debian release. However, since different releases will have different sets of
packages available, you'll need to either modify your stages accordingly, or
checkout the appropriate branch. For example, if you'd like to build a
- `buster` image, you should do so from the `buster` branch.
+ `bullseye` image, you should do so from the `bullseye` branch.
* `APT_PROXY` (Default: unset)
@@ -97,12 +77,6 @@ The following environment variables are supported:
will not be included in the image, making it safe to use an `apt-cacher` or
similar package for development.
- If you have Docker installed, you can set up a local apt caching proxy to
- like speed up subsequent builds like this:
-
- docker-compose up -d
- echo 'APT_PROXY=http://172.17.0.1:3142' >> config
-
* `BASE_DIR` (Default: location of `build.sh`)
**CAUTION**: Currently, changing this value will probably break build.sh
@@ -110,7 +84,7 @@ The following environment variables are supported:
Top-level directory for `pi-gen`. Contains stage directories, build
scripts, and by default both work and deployment directories.
- * `WORK_DIR` (Default: `"$BASE_DIR/work"`)
+ * `WORK_DIR` (Default: `$BASE_DIR/work`)
Directory in which `pi-gen` builds the target system. This value can be
changed if you have a suitably large, fast storage location for stages to
@@ -120,7 +94,7 @@ The following environment variables are supported:
**CAUTION**: If your working directory is on an NTFS partition you probably won't be able to build: make sure this is a proper Linux filesystem.
- * `DEPLOY_DIR` (Default: `"$BASE_DIR/deploy"`)
+ * `DEPLOY_DIR` (Default: `$BASE_DIR/deploy`)
Output directory for target system images and NOOBS bundles.
@@ -147,20 +121,20 @@ The following environment variables are supported:
information on this. Usually 0 is no compression but very fast, up to 9 with
the best compression but very slow ).
- * `USE_QEMU` (Default: `"0"`)
+ * `USE_QEMU` (Default: `0`)
Setting to '1' enables the QEMU mode - creating an image that can be mounted via QEMU for an emulated
environment. These images include "-qemu" in the image file name.
- * `LOCALE_DEFAULT` (Default: "en_GB.UTF-8" )
+ * `LOCALE_DEFAULT` (Default: 'en_GB.UTF-8' )
Default system locale.
- * `TARGET_HOSTNAME` (Default: "raspberrypi" )
+ * `TARGET_HOSTNAME` (Default: 'raspberrypi' )
Setting the hostname to the specified value.
- * `KEYBOARD_KEYMAP` (Default: "gb" )
+ * `KEYBOARD_KEYMAP` (Default: 'gb' )
Default keyboard keymap.
@@ -168,7 +142,7 @@ The following environment variables are supported:
keyboard-configuration` and look at the
`keyboard-configuration/xkb-keymap` value.
- * `KEYBOARD_LAYOUT` (Default: "English (UK)" )
+ * `KEYBOARD_LAYOUT` (Default: 'English (UK)' )
Default keyboard layout.
@@ -176,7 +150,7 @@ The following environment variables are supported:
keyboard-configuration` and look at the
`keyboard-configuration/variant` value.
- * `TIMEZONE_DEFAULT` (Default: "Europe/London" )
+ * `TIMEZONE_DEFAULT` (Default: 'Europe/London' )
Default keyboard layout.
@@ -231,10 +205,10 @@ The following environment variables are supported:
If set, then instead of working through the numeric stages in order, this list will be followed. For example setting to `"stage0 stage1 mystage stage2"` will run the contents of `mystage` before stage2. Note that quotes are needed around the list. An absolute or relative path can be given for stages outside the pi-gen directory.
-A simple example for building Raspbian:
+A simple example for building Raspberry Pi OS:
```bash
-IMG_NAME='Raspbian'
+IMG_NAME='raspios'
```
The config file can also be specified on the command line as an argument the `build.sh` or `build-docker.sh` scripts.
@@ -249,17 +223,17 @@ This is parsed after `config` so can be used to override values set there.
The following process is followed to build images:
- * Loop through all of the stage directories in alphanumeric order
+ * Interate through all of the stage directories in alphanumeric order
- * Move on to the next directory if this stage directory contains a file called
+ * Bypass a stage directory if it contains a file called
"SKIP"
* Run the script ```prerun.sh``` which is generally just used to copy the build
directory between stages.
- * In each stage directory loop through each subdirectory and then run each of the
- install scripts it contains, again in alphanumeric order. These need to be named
- with a two digit padded number at the beginning.
+ * In each stage directory iterate through each subdirectory and then run each of the
+ install scripts it contains, again in alphanumeric order. **These need to be named
+ with a two digit padded number at the beginning.**
There are a number of different files and directories which can be used to
control different parts of the build process:
@@ -294,7 +268,7 @@ It is recommended to examine build.sh for finer details.
Docker can be used to perform the build inside a container. This partially isolates
the build from the host system, and allows using the script on non-debian based
-systems (e.g. Fedora Linux). The isolate is not complete due to the need to use
+systems (e.g. Fedora Linux). The isolation is not complete due to the need to use
some kernel level services for arm emulation (binfmt) and loop devices (losetup).
To build:
@@ -307,7 +281,7 @@ vi config # Edit your config file. See above.
If everything goes well, your finished image will be in the `deploy/` folder.
You can then remove the build container with `docker rm -v pigen_work`
-If something breaks along the line, you can edit the corresponding scripts, and
+If you encounter errors during the build, you can edit the corresponding scripts, and
continue:
```bash
@@ -351,43 +325,36 @@ maintenance and allows for more easy customization.
`debootstrap`, which creates a minimal filesystem suitable for use as a
base.tgz on Debian systems. This stage also configures apt settings and
installs `raspberrypi-bootloader` which is missed by debootstrap. The
- minimal core is installed but not configured, and the system will not quite
- boot yet.
+ minimal core is installed but not configured. As a result, this stage will not boot.
- **Stage 1** - truly minimal system. This stage makes the system bootable by
installing system files like `/etc/fstab`, configures the bootloader, makes
the network operable, and installs packages like raspi-config. At this
stage the system should boot to a local console from which you have the
means to perform basic tasks needed to configure and install the system.
- This is as minimal as a system can possibly get, and its arguably not
- really usable yet in a traditional sense yet. Still, if you want minimal,
- this is minimal and the rest you could reasonably do yourself as sysadmin.
- - **Stage 2** - lite system. This stage produces the Raspbian-Lite image. It
- installs some optimized memory functions, sets timezone and charmap
+ - **Stage 2** - lite system. This stage produces the Raspberry Pi OS Lite image.
+ Stage 2 installs some optimized memory functions, sets timezone and charmap
defaults, installs fake-hwclock and ntp, wireless LAN and bluetooth support,
dphys-swapfile, and other basics for managing the hardware. It also
creates necessary groups and gives the pi user access to sudo and the
standard console hardware permission groups.
- There are a few tools that may not make a whole lot of sense here for
- development purposes on a minimal system such as basic Python and Lua
- packages as well as the `build-essential` package. They are lumped right
- in with more essential packages presently, though they need not be with
- pi-gen. These are understandable for Raspbian's target audience, but if
- you were looking for something between truly minimal and Raspbian-Lite,
- here's where you start trimming.
+ Note: Raspberry Pi OS Lite contains a number of tools for development,
+ including `Python`, `Lua` and the `build-essential` package. If you are
+ creating an image to deploy in products, be sure to remove extraneous development
+ tools before deployment.
- **Stage 3** - desktop system. Here's where you get the full desktop system
- with X11 and LXDE, web browsers, git for development, Raspbian custom UI
+ with X11 and LXDE, web browsers, git for development, Raspberry Pi OS custom UI
enhancements, etc. This is a base desktop system, with some development
tools installed.
- - **Stage 4** - Normal Raspbian image. System meant to fit on a 4GB card. This is the
- stage that installs most things that make Raspbian friendly to new
- users like system documentation.
+ - **Stage 4** - Normal Raspberry Pi OS image. System meant to fit on a 4GB card.
+ This is the stage that installs most things that make Raspberry Pi OS friendly
+ to new users - e.g. system documentation.
- - **Stage 5** - The Raspbian Full image. More development
+ - **Stage 5** - The Raspberry Pi OS Full image. More development
tools, an email client, learning tools like Scratch, specialized packages
like sonic-pi, office productivity, etc.
@@ -402,7 +369,7 @@ to `./stage2` (if building a minimal system).
```bash
# Example for building a lite system
-echo "IMG_NAME='Raspbian'" > config
+echo "IMG_NAME='raspios'" > config
touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
sudo ./build.sh # or ./build-docker.sh
@@ -429,71 +396,6 @@ follows:
* Once you're happy with the image you can remove the SKIP_IMAGES files and
export your image to test
-# Regarding Qcow2 image building
-
-### Get infos about the image in use
-
-If you issue the two commands shown in the example below in a second command shell while a build
-is running you can find out, which network block device is currently being used and which qcow2 image
-is bound to it.
-
-Example:
-
-```bash
-root@build-machine:~/$ lsblk | grep nbd
-nbd1 43:32 0 10G 0 disk
-├─nbd1p1 43:33 0 10G 0 part
-└─nbd1p1 253:0 0 10G 0 part
-
-root@build-machine:~/$ ps xa | grep qemu-nbd
- 2392 pts/6 S+ 0:00 grep --color=auto qemu-nbd
-31294 ? Ssl 0:12 qemu-nbd --discard=unmap -c /dev/nbd1 image-stage4.qcow2
-```
-
-Here you can see, that the qcow2 image `image-stage4.qcow2` is currently connected to `/dev/nbd1` with
-the associated partition map `/dev/mapper/nbd1p1`. Don't worry that `lsblk` shows two entries. It is totally fine, because the device map is accessible via `/dev/mapper/nbd1p1` and also via `/dev/dm-0`. This is all part of the device mapper functionality of the kernel. See `dmsetup` for further information.
-
-### Mount a qcow2 image
-
-If you want to examine the content of a a single stage, you can simply mount the qcow2 image found in the `WORK_DIR` directory with the tool `./imagetool.sh`.
-
-See `./imagetool.sh -h` for further details on how to use it.
-
-### Disconnect an image if something went wrong
-
-It can happen, that your build stops in case of an error. Normally `./build.sh` should handle image disconnection appropriately, but in rare cases, especially during a Docker build, this may not work as expected. If that happens, starting a new build will fail and you may have to disconnect the image and/or device yourself.
-
-A typical message indicating that there are some orphaned device mapper entries is this:
-
-```
-Failed to set NBD socket
-Disconnect client, due to: Unexpected end-of-file before all bytes were read
-```
-
-If that happens go through the following steps:
-
-1. First, check if the image is somehow mounted to a directory entry and umount it as you would any other block device, like i.e. a hard disk or USB stick.
-
-2. Second, to disconnect an image from `qemu-nbd`, the QEMU Disk Network Block Device Server, issue the following command (be sure to change the device name to the one actually used):
-
- ```bash
- sudo qemu-nbd -d /dev/nbd1
- ```
-
- Note: if you use Docker build, normally no active `qemu-nbd` process exists anymore as it will be terminated when the Docker container stops.
-
-3. To disconnect a device partition map from the network block device, execute:
-
- ```bash
- sudo kpartx -d /dev/nbd1
- or
- sudo ./imagetool.sh --cleanup
- ```
-
- Note: The `imagetool.sh` command will cleanup any /dev/nbdX that is not connected to a running `qemu-nbd` daemon. Be careful if you use network block devices for other tasks utilizing NBDs on your build machine as well.
-
-Now you should be able to start a new build without running into troubles again. Most of the time, especially when using Docker build, you will only need no. 3 to get everything up and running again.
-
# Troubleshooting
## `64 Bit Systems`
diff --git a/build-docker.sh b/build-docker.sh
index 219198b..a12ee69 100755
--- a/build-docker.sh
+++ b/build-docker.sh
@@ -141,9 +141,6 @@ time ${DOCKER} run \
$DOCKER_CMDLINE_PRE \
--name "${DOCKER_CMDLINE_NAME}" \
--privileged \
- --cap-add=ALL \
- -v /dev:/dev \
- -v /lib/modules:/lib/modules \
${PIGEN_DOCKER_OPTS} \
--volume "${CONFIG_FILE}":/config:ro \
-e "GIT_HASH=${GIT_HASH}" \
diff --git a/build.sh b/build.sh
index 70d4a78..699b1de 100755
--- a/build.sh
+++ b/build.sh
@@ -24,11 +24,6 @@ EOF
on_chroot << EOF
apt-get -o Acquire::Retries=3 install --no-install-recommends -y $PACKAGES
EOF
- if [ "${USE_QCOW2}" = "1" ]; then
- on_chroot << EOF
-apt-get clean
-EOF
- fi
fi
log "End ${SUB_STAGE_DIR}/${i}-packages-nr"
fi
@@ -39,11 +34,6 @@ EOF
on_chroot << EOF
apt-get -o Acquire::Retries=3 install -y $PACKAGES
EOF
- if [ "${USE_QCOW2}" = "1" ]; then
- on_chroot << EOF
-apt-get clean
-EOF
- fi
fi
log "End ${SUB_STAGE_DIR}/${i}-packages"
fi
@@ -102,16 +92,7 @@ run_stage(){
STAGE_WORK_DIR="${WORK_DIR}/${STAGE}"
ROOTFS_DIR="${STAGE_WORK_DIR}"/rootfs
- if [ "${USE_QCOW2}" = "1" ]; then
- if [ ! -f SKIP ]; then
- load_qimage
- fi
- else
- # make sure we are not umounting during export-image stage
- if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
- unmount "${WORK_DIR}/${STAGE}"
- fi
- fi
+ unmount "${WORK_DIR}/${STAGE}"
if [ ! -f SKIP_IMAGES ]; then
if [ -f "${STAGE_DIR}/EXPORT_IMAGE" ]; then
@@ -119,7 +100,7 @@ run_stage(){
fi
fi
if [ ! -f SKIP ]; then
- if [ "${CLEAN}" = "1" ] && [ "${USE_QCOW2}" = "0" ] ; then
+ if [ "${CLEAN}" = "1" ]; then
if [ -d "${ROOTFS_DIR}" ]; then
rm -rf "${ROOTFS_DIR}"
fi
@@ -136,14 +117,7 @@ run_stage(){
done
fi
- if [ "${USE_QCOW2}" = "1" ]; then
- unload_qimage
- else
- # make sure we are not umounting during export-image stage
- if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
- unmount "${WORK_DIR}/${STAGE}"
- fi
- fi
+ unmount "${WORK_DIR}/${STAGE}"
PREV_STAGE="${STAGE}"
PREV_STAGE_DIR="${STAGE_DIR}"
@@ -187,9 +161,16 @@ do
done
term() {
- if [ "${USE_QCOW2}" = "1" ]; then
- log "Unloading image"
- unload_qimage
+ if [ "$?" -ne 0 ]; then
+ log "Build failed"
+ else
+ log "Build finished"
+ fi
+ unmount "${STAGE_WORK_DIR}"
+ if [ "$STAGE" = "export-image" ]; then
+ for img in "${STAGE_WORK_DIR}/"*.img; do
+ unmount_image "$img"
+ done
fi
}
@@ -199,10 +180,9 @@ export PI_GEN=${PI_GEN:-pi-gen}
export PI_GEN_REPO=${PI_GEN_REPO:-https://github.com/RPi-Distro/pi-gen}
export PI_GEN_RELEASE=${PI_GEN_RELEASE:-Raspberry Pi reference}
-if [ -z "${IMG_NAME}" ]; then
- echo "IMG_NAME not set" 1>&2
- exit 1
-fi
+export ARCH=arm64
+export RELEASE=${RELEASE:-bookworm} # Don't forget to update stage0/prerun.sh
+export IMG_NAME="${IMG_NAME:-raspios-$RELEASE-$ARCH}"
export USE_QEMU="${USE_QEMU:-0}"
export IMG_DATE="${IMG_DATE:-"$(date +%Y-%m-%d)"}"
@@ -230,7 +210,6 @@ export TARGET_HOSTNAME=${TARGET_HOSTNAME:-raspberrypi}
export FIRST_USER_NAME=${FIRST_USER_NAME:-pi}
export FIRST_USER_PASS
export DISABLE_FIRST_BOOT_USER_RENAME=${DISABLE_FIRST_BOOT_USER_RENAME:-0}
-export RELEASE=${RELEASE:-bookworm} # Don't forget to update stage0/prerun.sh
export WPA_COUNTRY
export ENABLE_SSH="${ENABLE_SSH:-0}"
export PUBKEY_ONLY_SSH="${PUBKEY_ONLY_SSH:-0}"
@@ -247,7 +226,6 @@ export GIT_HASH=${GIT_HASH:-"$(git rev-parse HEAD)"}
export PUBKEY_SSH_FIRST_USER
export CLEAN
-export IMG_NAME
export APT_PROXY
export STAGE
@@ -273,24 +251,21 @@ source "${SCRIPT_DIR}/common"
# shellcheck source=scripts/dependencies_check
source "${SCRIPT_DIR}/dependencies_check"
-export NO_PRERUN_QCOW2="${NO_PRERUN_QCOW2:-1}"
-export USE_QCOW2="${USE_QCOW2:-0}"
-export BASE_QCOW2_SIZE=${BASE_QCOW2_SIZE:-12G}
-source "${SCRIPT_DIR}/qcow2_handling"
-if [ "${USE_QCOW2}" = "1" ]; then
- NO_PRERUN_QCOW2=1
-else
- NO_PRERUN_QCOW2=0
-fi
-
-export NO_PRERUN_QCOW2="${NO_PRERUN_QCOW2:-1}"
-
if [ "$SETFCAP" != "1" ]; then
export CAPSH_ARG="--drop=cap_setfcap"
fi
dependencies_check "${BASE_DIR}/depends"
+echo "Checking native $ARCH executable support..."
+if ! arch-test -n "$ARCH"; then
+ echo "WARNING: Only a native build environment is supported. Checking emulated support..."
+ if ! arch-test "$ARCH"; then
+ echo "No fallback mechanism found. Ensure your OS has binfmt_misc support enabled and configured."
+ exit 1
+ fi
+fi
+
#check username is valid
if [[ ! "$FIRST_USER_NAME" =~ ^[a-z][-a-z0-9_]*$ ]]; then
echo "Invalid FIRST_USER_NAME: $FIRST_USER_NAME"
@@ -327,6 +302,7 @@ mkdir -p "${WORK_DIR}"
log "Begin ${BASE_DIR}"
STAGE_LIST=${STAGE_LIST:-${BASE_DIR}/stage*}
+export STAGE_LIST
for STAGE_DIR in $STAGE_LIST; do
STAGE_DIR=$(realpath "${STAGE_DIR}")
@@ -339,98 +315,22 @@ for EXPORT_DIR in ${EXPORT_DIRS}; do
# shellcheck source=/dev/null
source "${EXPORT_DIR}/EXPORT_IMAGE"
EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename "${EXPORT_DIR}")/rootfs
- if [ "${USE_QCOW2}" = "1" ]; then
- USE_QCOW2=0
- EXPORT_NAME="${IMG_FILENAME}${IMG_SUFFIX}"
- echo "------------------------------------------------------------------------"
- echo "Running export stage for ${EXPORT_NAME}"
- rm -f "${WORK_DIR}/export-image/${EXPORT_NAME}.img" || true
- rm -f "${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2" || true
- rm -f "${WORK_DIR}/${EXPORT_NAME}.img" || true
- rm -f "${WORK_DIR}/${EXPORT_NAME}.qcow2" || true
- EXPORT_STAGE=$(basename "${EXPORT_DIR}")
- for s in $STAGE_LIST; do
- TMP_LIST=${TMP_LIST:+$TMP_LIST }$(basename "${s}")
- done
- FIRST_STAGE=${TMP_LIST%% *}
- FIRST_IMAGE="image-${FIRST_STAGE}.qcow2"
-
- pushd "${WORK_DIR}" > /dev/null
- echo "Creating new base "${EXPORT_NAME}.qcow2" from ${FIRST_IMAGE}"
- cp "./${FIRST_IMAGE}" "${EXPORT_NAME}.qcow2"
-
- ARR=($TMP_LIST)
- # rebase stage images to new export base
- for CURR_STAGE in "${ARR[@]}"; do
- if [ "${CURR_STAGE}" = "${FIRST_STAGE}" ]; then
- PREV_IMG="${EXPORT_NAME}"
- continue
- fi
- echo "Rebasing image-${CURR_STAGE}.qcow2 onto ${PREV_IMG}.qcow2"
- qemu-img rebase -f qcow2 -u -b ${PREV_IMG}.qcow2 image-${CURR_STAGE}.qcow2
- if [ "${CURR_STAGE}" = "${EXPORT_STAGE}" ]; then
- break
- fi
- PREV_IMG="image-${CURR_STAGE}"
- done
-
- # commit current export stage into base export image
- echo "Committing image-${EXPORT_STAGE}.qcow2 to ${EXPORT_NAME}.qcow2"
- qemu-img commit -f qcow2 -p -b "${EXPORT_NAME}.qcow2" image-${EXPORT_STAGE}.qcow2
-
- # rebase stage images back to original first stage for easy re-run
- for CURR_STAGE in "${ARR[@]}"; do
- if [ "${CURR_STAGE}" = "${FIRST_STAGE}" ]; then
- PREV_IMG="image-${CURR_STAGE}"
- continue
- fi
- echo "Rebasing back image-${CURR_STAGE}.qcow2 onto ${PREV_IMG}.qcow2"
- qemu-img rebase -f qcow2 -u -b ${PREV_IMG}.qcow2 image-${CURR_STAGE}.qcow2
- if [ "${CURR_STAGE}" = "${EXPORT_STAGE}" ]; then
- break
- fi
- PREV_IMG="image-${CURR_STAGE}"
- done
- popd > /dev/null
-
- mkdir -p "${WORK_DIR}/export-image/rootfs"
- mv "${WORK_DIR}/${EXPORT_NAME}.qcow2" "${WORK_DIR}/export-image/"
- echo "Mounting image ${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2 to rootfs ${WORK_DIR}/export-image/rootfs"
- mount_qimage "${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2" "${WORK_DIR}/export-image/rootfs"
-
- CLEAN=0
- run_stage
- CLEAN=1
- USE_QCOW2=1
-
- else
- run_stage
- fi
+ run_stage
if [ "${USE_QEMU}" != "1" ]; then
if [ -e "${EXPORT_DIR}/EXPORT_NOOBS" ]; then
# shellcheck source=/dev/null
source "${EXPORT_DIR}/EXPORT_NOOBS"
STAGE_DIR="${BASE_DIR}/export-noobs"
- if [ "${USE_QCOW2}" = "1" ]; then
- USE_QCOW2=0
- run_stage
- USE_QCOW2=1
- else
- run_stage
- fi
+ run_stage
fi
fi
done
-if [ -x postrun.sh ]; then
+if [ -x "${BASE_DIR}/postrun.sh" ]; then
log "Begin postrun.sh"
cd "${BASE_DIR}"
./postrun.sh
log "End postrun.sh"
fi
-if [ "${USE_QCOW2}" = "1" ]; then
- unload_qimage
-fi
-
log "End ${BASE_DIR}"
diff --git a/depends b/depends
index db88171..acbb5cd 100644
--- a/depends
+++ b/depends
@@ -17,7 +17,6 @@ file
git
lsmod:kmod
bc
-qemu-nbd:qemu-utils
-kpartx
gpg
pigz
+arch-test
diff --git a/docker-compose.yml b/docker-compose.yml
deleted file mode 100644
index f733860..0000000
--- a/docker-compose.yml
+++ /dev/null
@@ -1,10 +0,0 @@
-version: '2'
-
-services:
- apt-cacher-ng:
- restart: unless-stopped
- image: sameersbn/apt-cacher-ng:latest
- ports:
- - "3142:3142"
- volumes:
- - ./apt-cacher-ng:/var/cache/apt-cacher-ng
diff --git a/export-image/04-set-partuuid/00-run.sh b/export-image/04-set-partuuid/00-run.sh
index 2694295..99500f3 100755
--- a/export-image/04-set-partuuid/00-run.sh
+++ b/export-image/04-set-partuuid/00-run.sh
@@ -1,17 +1,13 @@
#!/bin/bash -e
-if [ "${NO_PRERUN_QCOW2}" = "0" ]; then
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
- IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
+IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
- IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
+BOOT_PARTUUID="${IMGID}-01"
+ROOT_PARTUUID="${IMGID}-02"
- BOOT_PARTUUID="${IMGID}-01"
- ROOT_PARTUUID="${IMGID}-02"
-
- sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
- sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
-
- sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/firmware/cmdline.txt"
-fi
+sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
+sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/firmware/cmdline.txt"
diff --git a/export-image/05-finalise/01-run.sh b/export-image/05-finalise/01-run.sh
index 3b56b20..771aa7a 100755
--- a/export-image/05-finalise/01-run.sh
+++ b/export-image/05-finalise/01-run.sh
@@ -83,24 +83,17 @@ cp "$ROOTFS_DIR/etc/rpi-issue" "$INFO_FILE"
dpkg -l --root "$ROOTFS_DIR"
} >> "$INFO_FILE"
-mkdir -p "${DEPLOY_DIR}"
+ROOT_DEV="$(awk "\$2 == \"${ROOTFS_DIR}\" {print \$1}" /etc/mtab)"
-rm -f "${DEPLOY_DIR}/${ARCHIVE_FILENAME}${IMG_SUFFIX}.*"
-rm -f "${DEPLOY_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
+unmount "${ROOTFS_DIR}"
+zerofree "${ROOT_DEV}"
-mv "$INFO_FILE" "$DEPLOY_DIR/"
+unmount_image "${IMG_FILE}"
-if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
- ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
-
- unmount "${ROOTFS_DIR}"
- zerofree "${ROOT_DEV}"
+mkdir -p "${DEPLOY_DIR}"
- unmount_image "${IMG_FILE}"
-else
- unload_qimage
- make_bootable_image "${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.qcow2" "$IMG_FILE"
-fi
+rm -f "${DEPLOY_DIR}/${ARCHIVE_FILENAME}${IMG_SUFFIX}.*"
+rm -f "${DEPLOY_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
case "${DEPLOY_COMPRESSION}" in
zip)
@@ -121,3 +114,5 @@ none | *)
cp "$IMG_FILE" "$DEPLOY_DIR/"
;;
esac
+
+cp "$INFO_FILE" "$DEPLOY_DIR/"
diff --git a/export-image/prerun.sh b/export-image/prerun.sh
index d042c6a..6a5309a 100755
--- a/export-image/prerun.sh
+++ b/export-image/prerun.sh
@@ -1,68 +1,73 @@
#!/bin/bash -e
-if [ "${NO_PRERUN_QCOW2}" = "0" ]; then
- IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
-
- unmount_image "${IMG_FILE}"
-
- rm -f "${IMG_FILE}"
-
- rm -rf "${ROOTFS_DIR}"
- mkdir -p "${ROOTFS_DIR}"
-
- BOOT_SIZE="$((512 * 1024 * 1024))"
- ROOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot/firmware --block-size=1 | cut -f 1)
-
- # All partition sizes and starts will be aligned to this size
- ALIGN="$((4 * 1024 * 1024))"
- # Add this much space to the calculated file size. This allows for
- # some overhead (since actual space usage is usually rounded up to the
- # filesystem block size) and gives some free space on the resulting
- # image.
- ROOT_MARGIN="$(echo "($ROOT_SIZE * 0.2 + 200 * 1024 * 1024) / 1" | bc)"
-
- BOOT_PART_START=$((ALIGN))
- BOOT_PART_SIZE=$(((BOOT_SIZE + ALIGN - 1) / ALIGN * ALIGN))
- ROOT_PART_START=$((BOOT_PART_START + BOOT_PART_SIZE))
- ROOT_PART_SIZE=$(((ROOT_SIZE + ROOT_MARGIN + ALIGN - 1) / ALIGN * ALIGN))
- IMG_SIZE=$((BOOT_PART_START + BOOT_PART_SIZE + ROOT_PART_SIZE))
-
- truncate -s "${IMG_SIZE}" "${IMG_FILE}"
-
- parted --script "${IMG_FILE}" mklabel msdos
- parted --script "${IMG_FILE}" unit B mkpart primary fat32 "${BOOT_PART_START}" "$((BOOT_PART_START + BOOT_PART_SIZE - 1))"
- parted --script "${IMG_FILE}" unit B mkpart primary ext4 "${ROOT_PART_START}" "$((ROOT_PART_START + ROOT_PART_SIZE - 1))"
-
- echo "Creating loop device..."
- cnt=0
- until ensure_next_loopdev && LOOP_DEV="$(losetup --show --find --partscan "$IMG_FILE")"; do
- if [ $cnt -lt 5 ]; then
- cnt=$((cnt + 1))
- echo "Error in losetup. Retrying..."
- sleep 5
- else
- echo "ERROR: losetup failed; exiting"
- exit 1
- fi
- done
-
- ensure_loopdev_partitions "$LOOP_DEV"
- BOOT_DEV="${LOOP_DEV}p1"
- ROOT_DEV="${LOOP_DEV}p2"
-
- ROOT_FEATURES="^huge_file"
- for FEATURE in 64bit; do
- if grep -q "$FEATURE" /etc/mke2fs.conf; then
- ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
+IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
+
+unmount_image "${IMG_FILE}"
+
+rm -f "${IMG_FILE}"
+
+rm -rf "${ROOTFS_DIR}"
+mkdir -p "${ROOTFS_DIR}"
+
+BOOT_SIZE="$((512 * 1024 * 1024))"
+ROOT_SIZE=$(du -x --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot/firmware --block-size=1 | cut -f 1)
+
+# All partition sizes and starts will be aligned to this size
+ALIGN="$((4 * 1024 * 1024))"
+# Add this much space to the calculated file size. This allows for
+# some overhead (since actual space usage is usually rounded up to the
+# filesystem block size) and gives some free space on the resulting
+# image.
+ROOT_MARGIN="$(echo "($ROOT_SIZE * 0.2 + 200 * 1024 * 1024) / 1" | bc)"
+
+BOOT_PART_START=$((ALIGN))
+BOOT_PART_SIZE=$(((BOOT_SIZE + ALIGN - 1) / ALIGN * ALIGN))
+ROOT_PART_START=$((BOOT_PART_START + BOOT_PART_SIZE))
+ROOT_PART_SIZE=$(((ROOT_SIZE + ROOT_MARGIN + ALIGN - 1) / ALIGN * ALIGN))
+IMG_SIZE=$((BOOT_PART_START + BOOT_PART_SIZE + ROOT_PART_SIZE))
+
+truncate -s "${IMG_SIZE}" "${IMG_FILE}"
+
+parted --script "${IMG_FILE}" mklabel msdos
+parted --script "${IMG_FILE}" unit B mkpart primary fat32 "${BOOT_PART_START}" "$((BOOT_PART_START + BOOT_PART_SIZE - 1))"
+parted --script "${IMG_FILE}" unit B mkpart primary ext4 "${ROOT_PART_START}" "$((ROOT_PART_START + ROOT_PART_SIZE - 1))"
+
+echo "Creating loop device..."
+cnt=0
+until ensure_next_loopdev && LOOP_DEV="$(losetup --show --find --partscan "$IMG_FILE")"; do
+ if [ $cnt -lt 5 ]; then
+ cnt=$((cnt + 1))
+ echo "Error in losetup. Retrying..."
+ sleep 5
+ else
+ echo "ERROR: losetup failed; exiting"
+ exit 1
fi
- done
- mkdosfs -n bootfs -F 32 -s 4 -v "$BOOT_DEV" > /dev/null
- mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
+done
- mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
- mkdir -p "${ROOTFS_DIR}/boot/firmware"
- mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot/firmware" -t vfat
+ensure_loopdev_partitions "$LOOP_DEV"
+BOOT_DEV="${LOOP_DEV}p1"
+ROOT_DEV="${LOOP_DEV}p2"
+
+ROOT_FEATURES="^huge_file"
+for FEATURE in 64bit; do
+if grep -q "$FEATURE" /etc/mke2fs.conf; then
+ ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
+fi
+done
- rsync -aHAXx --exclude /var/cache/apt/archives --exclude /boot/firmware "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"
- rsync -rtx "${EXPORT_ROOTFS_DIR}/boot/firmware/" "${ROOTFS_DIR}/boot/firmware/"
+if [ "$BOOT_SIZE" -lt 134742016 ]; then
+ FAT_SIZE=16
+else
+ FAT_SIZE=32
fi
+
+mkdosfs -n bootfs -F "$FAT_SIZE" -s 4 -v "$BOOT_DEV" > /dev/null
+mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
+
+mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
+mkdir -p "${ROOTFS_DIR}/boot/firmware"
+mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot/firmware" -t vfat
+
+rsync -aHAXx --exclude /var/cache/apt/archives --exclude /boot/firmware "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"
+rsync -rtx "${EXPORT_ROOTFS_DIR}/boot/firmware/" "${ROOTFS_DIR}/boot/firmware/"
diff --git a/export-noobs/00-release/00-run.sh b/export-noobs/00-release/00-run.sh
index bfaea9f..513cbfd 100755
--- a/export-noobs/00-release/00-run.sh
+++ b/export-noobs/00-release/00-run.sh
@@ -41,8 +41,4 @@ sed "${NOOBS_DIR}/os.json" -i -e "s|KERNEL|$(cat "${STAGE_WORK_DIR}/kernel_versi
sed "${NOOBS_DIR}/release_notes.txt" -i -e "s|UNRELEASED|${IMG_DATE}|"
-if [ "${USE_QCOW2}" = "1" ]; then
- mv "${NOOBS_DIR}" "${DEPLOY_DIR}/"
-else
- cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"
-fi
+cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"
diff --git a/export-noobs/00-release/files/release_notes.txt b/export-noobs/00-release/files/release_notes.txt
index d305e33..3512df2 100644
--- a/export-noobs/00-release/files/release_notes.txt
+++ b/export-noobs/00-release/files/release_notes.txt
@@ -1,4 +1,79 @@
UNRELEASED:
+ * pipanel - allow customisation of more than 2 desktops
+ * pipanel - add customisation for labwc
+ * gui-pkinst - add whitelist to restrict installation to specified packages only
+ * pixflat-theme - add theme settings for labwc
+ * pishutdown - revert to original use of pkill to close desktop
+ * piclone - fix for potential buffer overflow vulnerability (that would never have actually happened…)
+ * lp-connection-editor - fix dialog icons on taskbar
+ * rp-prefapps - add Raspberry Pi Connect; remove SmartSim
+ * piwiz - add page to enable / disable Raspberry Pi Connect
+ * wf-panel-pi - constrain main menu to fit on small screens
+ * wf-panel-pi - fix dialog icons on taskbar
+ * wf-panel-pi - fix keyboard handling and icon highlighting for taskbar buttons
+ * raspberrypi-ui-mods - add configuration for labwc
+ * raspberrypi-ui-mods - add support for new touchscreens
+ * raspberrypi-ui-mods - systemd-inhibit used to override hardware power key on Pi 5
+ * rc-gui - add configuration of alternate keyboard layout
+ * rc-gui - add switching for Raspberry Pi Connect
+ * arandr - add brightness control for DSI displays
+ * arandr - more reliable method to detect virtual displays
+ * raspi-config - add setting of keyboard options
+ * raspi-config - add setting of PCIe speed
+ * raspi-config - add switching for Raspberry Pi Connect
+ * wayvnc - better handling for virtual displays
+ * wayvnc - improved encryption support
+ * GTK-3 - add keyboard shortcuts in combo boxes
+ * pcmanfm - allow customisation of more than 2 desktops
+ * pcmanfm - fix bug causing crash and inconsistent behaviour on certain drag and drop operations
+ * raspberrypi-sys-mods - add udev rule to allow backlight change
+ * raspberrypi-sys-mods - increase swapfile size
+ * raspberrypi-sys-mods - remove symlinks from paths in initramfs scripts
+ * wayfire - fix for crash when opening multiple Xwayland windows
+ * wayfire - fix for touchscreen bug when touching areas without windows
+ * labwc compositor installed as an alternative to wayfire; can be enabled in raspi-config
+ * various small bug fixes and tweaks
+ * Chromium updated to 125.0.6422.133
+ * Firefox updated to 126.0
+ * Raspberry Pi firmware 3590de0c181d433af368a95f15bc480bdaff8b47
+ * Linux kernel 6.6.31 - c1432b4bae5b6582f4d32ba381459f33c34d1424
+2024-03-15:
+ * Audio streams will now not be interrupted when other audio devices are connected or disconnected
+ * Keyboard shortcut to install Orca no longer prompts for password, and will now wait for clock synchronisation rather than failing silently
+ * Orca screen reader updated to version 45 with various additional small bug fixes
+ * Obsolete fbturbo video driver removed
+ * Bug fix - saved display resolution settings not reloading under X
+ * Raspberry Pi firmware 6e0ae774407d68659b50cfbeb9f493ed16718866
+ * Linux kernel 6.6.20 - 6f16847710cc0502450788b9f12f0a14d3429668
+2024-03-12:
+ * Added setting of headless resolution to Screen Configuration
+ * Removed setting of headless resolution for Wayland from Raspberry Pi Configuration
+ * Improved handling of power button on Raspberry Pi 5
+ * Popover windows from taskbar replaced with conventional windows
+ * Shutdown assistant now closes all user processes when logging out
+ * Wayvnc updated to improve compatibility with various VNC clients
+ * Wayvnc now controlled by systemd
+ * Audio icon on taskbar hidden if no audio devices connected
+ * Alternative mouse cursor shown during drag-and-drop operations
+ * raspi-config now allows EEPROM to be upgraded
+ * Speed improvement when opening bluetooth and network menus
+ * Tweaks to display of some widgets under dark theme
+ * Improved compatibility with alternative window managers
+ * Bug fix - prevent multiple file manager confirm dialogs being overlaid
+ * Bug fix - drag-and-drop in file manager causing incorrect files to move
+ * Bug fix - memory leaks in volume and bluetooth menus
+ * Bug fix - GPU load sometimes not correctly reported in plugin and task manager
+ * Bug fix - crash when closing windows with non-GTK headerbars
+ * Bug fix - spurious button hover highlights on touchscreens
+ * Bug fix - windows on other monitors being hidden from taskbar
+ * Bug fix - corrected power monitoring brownout detection
+ * Bug fix - wayfire keyboard layout settings sometimes not loading
+ * Removed fbturbo xorg video driver as it is no longer useful
+ * Chromium updated to 122.0.6261.89
+ * Firefox updated to 123.0
+ * Raspberry Pi firmware 6e0ae774407d68659b50cfbeb9f493ed16718866
+ * Linux kernel 6.6.20 - 6f16847710cc0502450788b9f12f0a14d3429668
+2023-12-11:
* Fix Raspberry Pi Imager's WLAN configuration for lite images
2023-12-05:
* Serial port switching in rc_gui and raspi-config modified to support Raspberry Pi 5
diff --git a/export-noobs/prerun.sh b/export-noobs/prerun.sh
index 80cbb8e..8e55c9e 100755
--- a/export-noobs/prerun.sh
+++ b/export-noobs/prerun.sh
@@ -41,8 +41,4 @@ bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs/boot" -cpf -
umount "${STAGE_WORK_DIR}/rootfs/boot"
bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs" --one-file-system -cpf - . | xz -T0 > "${NOOBS_DIR}/root.tar.xz"
-if [ "${USE_QCOW2}" = "1" ]; then
- rm "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
-fi
-
unmount_image "${IMG_FILE}"
diff --git a/imagetool.sh b/imagetool.sh
deleted file mode 100755
index 002b50b..0000000
--- a/imagetool.sh
+++ /dev/null
@@ -1,114 +0,0 @@
-#!/bin/bash
-
-if [ "$(id -u)" != "0" ]; then
- echo "Please run as root" 1>&2
- exit 1
-fi
-
-progname=$(basename $0)
-
-function usage()
-{
- cat << HEREDOC
-
-Usage:
- Mount Image : $progname [--mount] [--image-name ] [--mount-point ]
- Umount Image: $progname [--umount] [--mount-point ]
- Cleanup NBD : $progname [--cleanup]
-
- arguments:
- -h, --help show this help message and exit
- -c, --cleanup cleanup orphaned device mappings
- -m, --mount mount image
- -u, --umount umount image
- -i, --image-name path to qcow2 image
- -p, --mount-point mount point for image
-
- This tool will use /dev/nbd1 as default for mounting an image. If you want to use another device, execute like this:
- NBD_DEV=/dev/nbd2 ./$progname --mount --image-name --mount-point
-
-HEREDOC
-}
-
-MOUNT=0
-UMOUNT=0
-IMAGE=""
-MOUNTPOINT=""
-
-nbd_cleanup() {
- DEVS="$(lsblk | grep nbd | grep disk | cut -d" " -f1)"
- if [ ! -z "${DEVS}" ]; then
- for d in $DEVS; do
- if [ ! -z "${d}" ]; then
- QDEV="$(ps xa | grep $d | grep -v grep)"
- if [ -z "${QDEV}" ]; then
- kpartx -d /dev/$d && echo "Unconnected device map removed: /dev/$d"
- fi
- fi
- done
- fi
-}
-
-# As long as there is at least one more argument, keep looping
-while [[ $# -gt 0 ]]; do
- key="$1"
- case "$key" in
- -h|--help)
- usage
- exit
- ;;
- -c|--cleanup)
- nbd_cleanup
- ;;
- -m|--mount)
- MOUNT=1
- ;;
- -u|--umount)
- UMOUNT=1
- ;;
- -i|--image-name)
- shift
- IMAGE="$1"
- ;;
- -p|--mount-point)
- shift
- MOUNTPOINT="$1"
- ;;
- *)
- echo "Unknown option '$key'"
- usage
- exit
- ;;
- esac
- # Shift after checking all the cases to get the next option
- shift
-done
-
-if [ "${MOUNT}" = "1" ] && [ "${UMOUNT}" = "1" ]; then
- usage
- echo "Concurrent mount options not possible."
- exit
-fi
-
-if [ "${MOUNT}" = "1" ] && ([ -z "${IMAGE}" ] || [ -z "${MOUNTPOINT}" ]); then
- usage
- echo "Can not mount image. Image path and/or mount point missing."
- exit
-fi
-
-if [ "${UMOUNT}" = "1" ] && [ -z "${MOUNTPOINT}" ]; then
- usage
- echo "Can not umount. Mount point parameter missing."
- exit
-fi
-
-export NBD_DEV="${NBD_DEV:-/dev/nbd1}"
-export MAP_BOOT_DEV=/dev/mapper/nbd1p1
-export MAP_ROOT_DEV=/dev/mapper/nbd1p2
-source scripts/qcow2_handling
-
-if [ "${MOUNT}" = "1" ]; then
- mount_qimage "${IMAGE}" "${MOUNTPOINT}"
-elif [ "${UMOUNT}" = "1" ]; then
- umount_qimage "${MOUNTPOINT}"
-fi
diff --git a/scripts/common b/scripts/common
index c290585..5a72ead 100644
--- a/scripts/common
+++ b/scripts/common
@@ -45,20 +45,26 @@ unmount(){
DIR=$1
fi
- while mount | grep -q "$DIR"; do
- local LOCS
- LOCS=$(mount | grep "$DIR" | cut -f 3 -d ' ' | sort -r)
- for loc in $LOCS; do
- umount "$loc"
- done
+ for i in {1..6}; do
+ if awk "\$2 ~ /^${DIR//\//\\/}/ {print \$2}" /etc/mtab | sort -r | xargs -r umount; then
+ break
+ elif [ "$i" -eq 6 ]; then
+ log "Failed to unmount ${DIR}. Do not try to delete this directory while it contains mountpoints!"
+ return 1
+ fi
+ log "Retrying ($i/5)..."
+ sleep 1
done
}
export -f unmount
unmount_image(){
- sync
- sleep 1
- LOOP_DEVICE=$(losetup --list | grep "$1" | cut -f1 -d' ')
+ if command -v udevadm >/dev/null 2>&1; then
+ udevadm settle 10
+ else
+ sleep 1
+ fi
+ LOOP_DEVICE=$(losetup -n -O NAME -j "$1")
if [ -n "$LOOP_DEVICE" ]; then
for part in "$LOOP_DEVICE"p*; do
if DIR=$(findmnt -n -o target -S "$part"); then
@@ -123,5 +129,7 @@ ensure_loopdev_partitions() {
mknod "/dev/$partition" b "${majmin%:*}" "${majmin#*:}"
fi
done
+ command -v udevadm >/dev/null 2>&1 || return 0
+ udevadm settle 10
}
export -f ensure_loopdev_partitions
diff --git a/scripts/qcow2_handling b/scripts/qcow2_handling
deleted file mode 100644
index 66708e7..0000000
--- a/scripts/qcow2_handling
+++ /dev/null
@@ -1,256 +0,0 @@
-#!/bin/bash
-
-# QCOW2 Routines
-
-export CURRENT_IMAGE
-export CURRENT_MOUNTPOINT
-
-export NBD_DEV
-export MAP_BOOT_DEV
-export MAP_ROOT_DEV
-
-# set in build.sh
-# should be fairly enough for the beginning
-# overwrite here by uncommenting following lines
-# BASE_QCOW2_SIZE=12G
-
-# find and initialize free block device nodes
-init_nbd() {
- modprobe nbd max_part=16
- if [ -z "${NBD_DEV}" ]; then
- for x in /sys/class/block/nbd* ; do
- S=`cat $x/size`
- if [ "$S" == "0" ] ; then
- NBD_DEV=/dev/$(basename $x)
- MAP_BOOT_DEV=/dev/mapper/$(basename $x)p1
- MAP_ROOT_DEV=/dev/mapper/$(basename $x)p2
- break
- fi
- done
- fi
-}
-export -f init_nbd
-
-# connect image to block device
-connect_blkdev() {
- init_nbd
- qemu-nbd --discard=unmap -c $NBD_DEV "$1"
- sync
- kpartx -as $NBD_DEV
- sync
- CURRENT_IMAGE="$1"
-}
-export -f connect_blkdev
-
-# disconnect image from block device
-disconnect_blkdev() {
- kpartx -d $NBD_DEV
- qemu-nbd -d $NBD_DEV
- NBD_DEV=
- MAP_BOOT_DEV=
- MAP_ROOT_DEV=
- CURRENT_IMAGE=
-}
-export -f disconnect_blkdev
-
-# mount qcow2 image: mount_image
-mount_qimage() {
- connect_blkdev "$1"
- mount -v -t ext4 $MAP_ROOT_DEV "$2"
- mkdir -p "${ROOTFS_DIR}/boot"
- mount -v -t vfat $MAP_BOOT_DEV "$2/boot"
- CURRENT_MOUNTPOINT="$2"
-}
-export -f mount_qimage
-
-# umount qcow2 image: umount_image
-umount_qimage() {
- sync
- #umount "$1/boot"
- while mount | grep -q "$1"; do
- local LOCS
- LOCS=$(mount | grep "$1" | cut -f 3 -d ' ' | sort -r)
- for loc in $LOCS; do
- echo "$loc"
- while mountpoint -q "$loc" && ! umount "$loc"; do
- sleep 0.1
- done
- done
- done
- CURRENT_MOUNTPOINT=
- disconnect_blkdev
-}
-export -f umount_qimage
-
-# create base image / backing image / mount image
-load_qimage() {
- if [ -z "${CURRENT_MOUNTPOINT}" ]; then
- if [ ! -d "${ROOTFS_DIR}" ]; then
- mkdir -p "${ROOTFS_DIR}";
- fi
-
- if [ "${CLEAN}" = "1" ] && [ -f "${WORK_DIR}/image-${STAGE}.qcow2" ]; then
- rm -f "${WORK_DIR}/image-${STAGE}.qcow2";
- fi
-
- if [ ! -f "${WORK_DIR}/image-${STAGE}.qcow2" ]; then
- pushd ${WORK_DIR} > /dev/null
- init_nbd
- if [ -z "${PREV_STAGE}" ]; then
- echo "Creating base image: image-${STAGE}.qcow2"
- # -o preallocation=falloc
- qemu-img create -f qcow2 image-${STAGE}.qcow2 $BASE_QCOW2_SIZE
- sync
- qemu-nbd --discard=unmap -c $NBD_DEV image-${STAGE}.qcow2
- sync
- sfdisk $NBD_DEV << EOF
-4MiB,250MiB,c,*
-254MiB,,83;
-EOF
- sync
- kpartx -as $NBD_DEV
- mkdosfs -n boot -F 32 -s 4 -v $MAP_BOOT_DEV
- mkfs.ext4 -L rootfs -O "^huge_file,^64bit" $MAP_ROOT_DEV
- sync
- else
- if [ ! -f "${WORK_DIR}/image-${PREV_STAGE}.qcow2" ]; then
- exit 1;
- fi
- echo "Creating backing image: image-${STAGE}.qcow2 <- ${WORK_DIR}/image-${PREV_STAGE}.qcow2"
- qemu-img create -f qcow2 \
- -o backing_file=${WORK_DIR}/image-${PREV_STAGE}.qcow2 \
- ${WORK_DIR}/image-${STAGE}.qcow2
- sync
- qemu-nbd --discard=unmap -c $NBD_DEV image-${STAGE}.qcow2
- sync
- kpartx -as $NBD_DEV
- fi
-
- mount -v -t ext4 $MAP_ROOT_DEV "${ROOTFS_DIR}"
- mkdir -p "${ROOTFS_DIR}/boot"
- mount -v -t vfat $MAP_BOOT_DEV "${ROOTFS_DIR}/boot"
- CURRENT_IMAGE=${WORK_DIR}/image-${STAGE}.qcow2
- CURRENT_MOUNTPOINT=${ROOTFS_DIR}
- popd > /dev/null
- else
- mount_qimage "${WORK_DIR}/image-${STAGE}.qcow2" "${ROOTFS_DIR}"
- fi
- echo "Current image in use: ${CURRENT_IMAGE} (MP: ${CURRENT_MOUNTPOINT})"
- fi
-}
-export -f load_qimage
-
-# umount current image and refresh mount point env var
-unload_qimage() {
- if [ ! -z "${CURRENT_MOUNTPOINT}" ]; then
- fstrim -v "${CURRENT_MOUNTPOINT}" || true
- umount_qimage "${CURRENT_MOUNTPOINT}"
- fi
-}
-export -f unload_qimage
-
-# based on: https://github.com/SirLagz/RaspberryPi-ImgAutoSizer
-# helper function for make_bootable_image, do not call directly
-function resize_qcow2() {
- if [ -z "$CALL_FROM_MBI" ]; then
- echo "resize_qcow2: cannot be called directly, use make_bootable_image instead"
- return 1
- fi
-
- # ROOT_MARGIN=$((800*1024*1024))
- ROOT_MARGIN=$((1*1024*1024))
- PARTED_OUT=`parted -s -m "$NBD_DEV" unit B print`
- PART_NO=`echo "$PARTED_OUT" | grep ext4 | awk -F: ' { print $1 } '`
- PART_START=`echo "$PARTED_OUT" | grep ext4 | awk -F: ' { print substr($2,1,length($2)-1) } '`
-
- e2fsck -y -f $MAP_ROOT_DEV || true
-
- DATA_SIZE=`resize2fs -P $MAP_ROOT_DEV | awk -F': ' ' { print $2 } '`
- BLOCK_SIZE=$(dumpe2fs -h $MAP_ROOT_DEV | grep 'Block size' | awk -F': ' ' { print $2 }')
- BLOCK_SIZE=${BLOCK_SIZE// /}
-
- let DATA_SIZE=$DATA_SIZE+$ROOT_MARGIN/$BLOCK_SIZE
- resize2fs -p $MAP_ROOT_DEV $DATA_SIZE
- sleep 1
-
- let PART_NEW_SIZE=$DATA_SIZE*$BLOCK_SIZE
- let PART_NEW_END=$PART_START+$PART_NEW_SIZE
- ACT1=`parted -s "$NBD_DEV" rm 2`
- ACT2=`parted -s "$NBD_DEV" unit B mkpart primary $PART_START $PART_NEW_END`
- NEW_IMG_SIZE=`parted -s -m "$NBD_DEV" unit B print free | tail -1 | awk -F: ' { print substr($2,1,length($2)-1) } '`
-}
-export -f resize_qcow2
-
-# create raw img from qcow2: make_bootable_image
-function make_bootable_image() {
-
- EXPORT_QCOW2="$1"
- EXPORT_IMAGE="$2"
-
- echo "Connect block device to source qcow2"
- connect_blkdev "${EXPORT_QCOW2}"
-
- echo "Resize fs and partition"
- CALL_FROM_MBI=1
- resize_qcow2
- sync
- CALL_FROM_MBI=
-
- echo "Disconnect block device"
- disconnect_blkdev
-
- if [ -z "$NEW_IMG_SIZE" ]; then
- echo "NEW_IMG_SIZE could not be calculated, cannot process image. Exit."
- exit 1
- fi
-
- echo "Shrinking qcow2 image"
- qemu-img resize --shrink "${EXPORT_QCOW2}" $NEW_IMG_SIZE
- sync
-
- echo "Convert qcow2 to raw image"
- qemu-img convert -f qcow2 -O raw "${EXPORT_QCOW2}" "${EXPORT_IMAGE}"
- sync
-
- echo "Get PARTUUIDs from image"
- IMGID="$(blkid -o value -s PTUUID "${EXPORT_IMAGE}")"
-
- BOOT_PARTUUID="${IMGID}-01"
- echo "Boot: $BOOT_PARTUUID"
- ROOT_PARTUUID="${IMGID}-02"
- echo "Root: $ROOT_PARTUUID"
-
- echo "Mount image"
- MOUNTROOT=${WORK_DIR}/tmpimage
- mkdir -p $MOUNTROOT
-
- MOUNTPT=$MOUNTROOT
- PARTITION=2
- mount "${EXPORT_IMAGE}" "$MOUNTPT" -o loop,offset=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*start=[ ]*//' | sed 's/,.*//'` * 512 ],sizelimit=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*size=[ ]*//' | sed 's/,.*//'` * 512 ] || exit 1
-
- MOUNTPT=$MOUNTROOT/boot
- PARTITION=1
- mount "${EXPORT_IMAGE}" "$MOUNTPT" -o loop,offset=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*start=[ ]*//' | sed 's/,.*//'` * 512 ],sizelimit=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*size=[ ]*//' | sed 's/,.*//'` * 512 ] || exit 1
-
- if [ ! -d "${MOUNTROOT}/root" ]; then
- echo "Image damaged or not mounted. Exit."
- exit 1
- fi
-
- echo "Setup PARTUUIDs"
- if [ ! -z "$BOOT_PARTUUID" ] && [ ! -z "$ROOT_PARTUUID" ]; then
- echo "Set UUIDs to make it bootable"
- sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${MOUNTROOT}/etc/fstab"
- sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${MOUNTROOT}/etc/fstab"
- sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${MOUNTROOT}/boot/cmdline.txt"
- fi
-
- echo "Umount image"
- sync
- umount "${MOUNTROOT}/boot" || exit 1
- umount "${MOUNTROOT}" || exit 1
-
- echo "Remove qcow2 export image"
- rm -f "${EXPORT_QCOW2}"
-}
-export -f make_bootable_image
diff --git a/stage-gl1/00-install-packages/00-packages b/stage-gl1/00-install-packages/00-packages
index 5ccad77..b87faea 100644
--- a/stage-gl1/00-install-packages/00-packages
+++ b/stage-gl1/00-install-packages/00-packages
@@ -1,7 +1,7 @@
# Absolute basics
python3 python-is-python3
-# we don't install things like python3-pip or many libraries here, because pip and apt don't
+# We don't install things like python3-pip or many libraries here, because pip and apt don't
# get along, so we'll set up a venv later.
# We'll set up a venv to get around this.
@@ -10,6 +10,10 @@ python3-venv
# We do install OS's opencv to make sure all the .so files are available.
python3-opencv
+# We need to make sure to install this before framegrab -- required to install the netifaces
+# package.
+python3-dev
+
# Some nice-to-have utilities
iotop nethogs net-tools telnet
git curl wget
diff --git a/stage0/prerun.sh b/stage0/prerun.sh
index da6f50a..4920b9b 100755
--- a/stage0/prerun.sh
+++ b/stage0/prerun.sh
@@ -5,6 +5,6 @@ if [ "$RELEASE" != "bookworm" ]; then
echo " Please check the relevant README.md section."
fi
-if [ ! -d "${ROOTFS_DIR}" ] || [ "${USE_QCOW2}" = "1" ]; then
+if [ ! -d "${ROOTFS_DIR}" ]; then
bootstrap ${RELEASE} "${ROOTFS_DIR}" http://deb.debian.org/debian/
fi
diff --git a/stage1/00-boot-files/files/config.txt b/stage1/00-boot-files/files/config.txt
index a1fdaeb..fecd26f 100644
--- a/stage1/00-boot-files/files/config.txt
+++ b/stage1/00-boot-files/files/config.txt
@@ -45,5 +45,7 @@ arm_boost=1
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1
-[all]
+[cm5]
+dtoverlay=dwc2,dr_mode=host
+[all]
diff --git a/stage2/01-sys-tweaks/00-debconf b/stage2/01-sys-tweaks/00-debconf
index c13e3b1..884db93 100644
--- a/stage2/01-sys-tweaks/00-debconf
+++ b/stage2/01-sys-tweaks/00-debconf
@@ -24,3 +24,5 @@ keyboard-configuration keyboard-configuration/ctrl_alt_bksp boolean true
# Keyboard layout:
# Choices: English (UK), English (UK) - English (UK\, Colemak), English (UK) - English (UK\, Dvorak with UK punctuation), English (UK) - English (UK\, Dvorak), English (UK) - English (UK\, Macintosh international), English (UK) - English (UK\, Macintosh), English (UK) - English (UK\, extended WinKeys), English (UK) - English (UK\, international with dead keys), Other
keyboard-configuration keyboard-configuration/variant select ${KEYBOARD_LAYOUT}
+# for internal use
+keyboard-configuration keyboard-configuration/optionscode string PLACEHOLDER
diff --git a/stage2/01-sys-tweaks/00-packages-nr b/stage2/01-sys-tweaks/00-packages-nr
index 6bdb600..15c0f66 100644
--- a/stage2/01-sys-tweaks/00-packages-nr
+++ b/stage2/01-sys-tweaks/00-packages-nr
@@ -1,3 +1,3 @@
cifs-utils
-libcamera-apps-lite
+rpicam-apps-lite
mkvtoolnix
diff --git a/stage2/01-sys-tweaks/00-patches/02-swap.diff b/stage2/01-sys-tweaks/00-patches/02-swap.diff
index 745a344..f542ec2 100644
--- a/stage2/01-sys-tweaks/00-patches/02-swap.diff
+++ b/stage2/01-sys-tweaks/00-patches/02-swap.diff
@@ -7,7 +7,7 @@ Index: jessie-stage2/rootfs/etc/dphys-swapfile
# set size to absolute value, leaving empty (default) then uses computed value
# you most likely don't want this, unless you have an special disk situation
-#CONF_SWAPSIZE=
-+CONF_SWAPSIZE=100
++CONF_SWAPSIZE=200
# set size to computed value, this times RAM size, dynamically adapts,
# guarantees that there is enough swap without wasting disk space on excess
diff --git a/stage2/01-sys-tweaks/01-run.sh b/stage2/01-sys-tweaks/01-run.sh
index 59988b2..897463e 100755
--- a/stage2/01-sys-tweaks/01-run.sh
+++ b/stage2/01-sys-tweaks/01-run.sh
@@ -67,3 +67,8 @@ usermod --pass='*' root
EOF
rm -f "${ROOTFS_DIR}/etc/ssh/"ssh_host_*_key*
+
+sed -i "s/PLACEHOLDER//" "${ROOTFS_DIR}/etc/default/keyboard"
+on_chroot << EOF
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure keyboard-configuration
+EOF
diff --git a/stage3/00-install-packages/00-packages b/stage3/00-install-packages/00-packages
index 6a48535..4d9649e 100644
--- a/stage3/00-install-packages/00-packages
+++ b/stage3/00-install-packages/00-packages
@@ -13,7 +13,7 @@ fonts-liberation2
obconf
arandr
libcamera-tools
-libcamera-apps
+rpicam-apps
python3-picamera2
python3-pyqt5
python3-opengl
diff --git a/stage4/00-install-packages/00-packages b/stage4/00-install-packages/00-packages
index 2307752..ebd1af4 100644
--- a/stage4/00-install-packages/00-packages
+++ b/stage4/00-install-packages/00-packages
@@ -18,4 +18,9 @@ piwiz
rp-prefapps
ffmpeg
vlc
+rpi-connect
rpi-imager
+labwc
+
+# ninja-build and openocd needed for vscode pico extension
+meson openocd
diff --git a/stage4/07-disable-pwr-button/00-run.sh b/stage4/07-disable-pwr-button/00-run.sh
deleted file mode 100755
index 44646bd..0000000
--- a/stage4/07-disable-pwr-button/00-run.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash -e
-
-sed -i 's/^.*HandlePowerKey=.*$/HandlePowerKey=ignore/' "${ROOTFS_DIR}/etc/systemd/logind.conf"
diff --git a/stage5/00-install-extras/00-packages b/stage5/00-install-extras/00-packages
index ad5dcad..85474dd 100644
--- a/stage5/00-install-extras/00-packages
+++ b/stage5/00-install-extras/00-packages
@@ -1,6 +1,5 @@
mu-editor
scratch nuscratch scratch3
-smartsim
wolfram-engine
claws-mail
realvnc-vnc-viewer