test/system: Update Ubuntu versions used for tests (16.04, 18.04, 20.04 -> 20.04, 22.04, 24.04) #1535
Conversation
|
Build succeeded. ✔️ unit-test SUCCESS in 4m 55s |
|
Build succeeded. ✔️ unit-test SUCCESS in 5m 19s |
There was a problem hiding this comment.
Thanks for working on this, @pennbauman ! My apologies for not responding sooner. I was caught up with getting a new release out.
I see that the pull request has two commits that are somewhat unrelated. So, let me start with a high level comment before getting into too many details.
The test/system: Fix create tests for host distro/version commit is certainly welcome. As we add built-in support for more and more distributions, the combinatorial explosion of host and container operating system pairs makes it important to run the tests on as many different host OSes as possible. Some months ago, I had made an aborted attempt at this. I am hoping that your effort will finally push this over the finishing line.
So, thanks for that.
The test/system: Update Ubuntu versions used for tests commit makes me curious. Were you experiencing failures from the tests for Ubuntus 16.04 and 18.04? The tests seem to be passing when run on the CI.
Other than the need to test as many host and container pairs as possible, there's one technical need to test containers with older operating systems. The toolbox(1) binary has a weird runtime ABI requirement. Binaries built on newer host OSes must run on containers with older OSes, so must binaries across different distributions. This is because toolbox(1) from the host is bind mounted into the containers as their entry point.
The toolbox-create(1) and toolbox-init-container(1) manuals talk about this in further detail. I once wrote a blog to describe the other options that we had explored.
This is generally unsupported - the inverse is supported.
So, we do various tricks when building the toolbox(1) binary to ensure that it will meet the above requirements. See commit 6063eb2 for the full details.
While this was implemented after discussion with the glibc developers, it's also not something that's officially supported by them. Toolbx is clearly pushing the boundaries here. So, it's important to keep testing against containers with older OSes to ensure that it still works. This is the main reason for still testing against fedora-toolbox:34.
Now, specifically about Ubuntus 16.04 and 18.04, I see that they are still covered by the Pro subscription. So, they are not completely unsupported. They could be useful as representatives of other similarly old OSes out there, like Debian. However, if they are causing some real problems that would be too expensive to solve given the benefits, then we can consider dropping them too.
The VERSION_ID field in os-release(5) is optional [1]. It's absent on Arch Linux, which follows a rolling-release model and uses the BUILD_ID field instead: BUILD_ID=rolling A subsequent commit will run the CI on Arch Linux. Hence, the code to get the default release from the host operating system can no longer assume the presence of the VERSION_ID field in os-release(5). Note that the arch-toolbox image is tagged with 'latest', in accordance with OCI conventions, not 'rolling' [2,3], which is the os-release(5) BUILD_ID. A similar change was made to toolbox(1) in commits 2ee82af and d14fd7b. [1] https://www.freedesktop.org/software/systemd/man/os-release.html [2] Commit 2568528 containers@2568528cb7f52663 containers#861 [3] Commit a4e5861 containers@a4e5861ae5c93625 containers#1308 containers#1438 containers#1535
This is a step towards running the CI on Arch Linux. containers#1438 containers#1535
The VERSION_ID field in os-release(5) is optional [1]. It's absent on Arch Linux, which follows a rolling-release model and uses the BUILD_ID field instead: BUILD_ID=rolling A subsequent commit will run the CI on Arch Linux. Hence, the code to get the default release from the host operating system can no longer assume the presence of the VERSION_ID field in os-release(5). Note that the arch-toolbox image is tagged with 'latest', in accordance with OCI conventions, not 'rolling' [2,3], which is the os-release(5) BUILD_ID. A similar change was made to toolbox(1) in commits 2ee82af and d14fd7b. [1] https://www.freedesktop.org/software/systemd/man/os-release.html [2] Commit 2568528 containers@2568528cb7f52663 containers#861 [3] Commit a4e5861 containers@a4e5861ae5c93625 containers#1308 containers#1438 containers#1535
This is a step towards running the CI on Arch Linux. containers#1438 containers#1535
test/system/libs/helpers.bash
Outdated
| @@ -551,6 +551,11 @@ function get_system_id() ( | |||
| function get_system_version() ( | |||
| local os_release | |||
|
|
|||
| if [ "$(get_system_id)" = "arch" ]; then | |||
| echo "latest" | |||
There was a problem hiding this comment.
I needed to solve the same problem that this hunk addresses for #1438 which reminded me again of this pull request.
Instead of calling get_system_id, we can directly use VERSION_ID because we also source os-release(5) here.
We need this commit for #1438 So, let me try to rebase this pull request, so that we can get this commit merged. |
debarshiray
force-pushed
the
ubuntu-tests-update
branch
from
43e6399 to
e805291
Compare
The Ubuntu versions used for testing are updated from 16.04, 18.04, and 20.04 to 20.04, 22.04, and 24.04. This brings all Ubuntu releases used into the current standard support window, and removes tests using Ubuntu 16.04 and 18.04 which have failed on every machine I've tried them on.
It may also/alternatively make sense to reduce the number of Ubuntu versions tested (probably to just 24.04), as no other distro is tested with multiple releases. Other distros could also have the versions used updated, for example Fedora 34 is used which is far out of date.