Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support building offline OS bundles for Oracle Linux 9.4 #1192

Merged
merged 6 commits into from
Oct 7, 2024
Merged

feat: Support building offline OS bundles for Oracle Linux 9.4 #1192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4fe250b
feat: Builds oracle-9.4 offline package
manoj-nutanix Sep 30, 2024
eb03f98
chore: removed debug statement and comments from GHA
manoj-nutanix Oct 3, 2024
1cfdfac
removes pre-commit changes
manoj-nutanix Oct 3, 2024
7372190
removes the non-offline oracle-9.4 tests from the GHA files
manoj-nutanix Oct 3, 2024
f59ff61
installs tar and container-selinux packages for oracle-9.4 offline/on…
manoj-nutanix Oct 4, 2024
f521da9
Adds online modes 'basic' and 'fips' for oracle-9.4 on vsphere to e2e…
manoj-nutanix Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/release-vsphere-template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ jobs:
buildConfig: "basic"
- os: "oracle 9.4"
buildConfig: "fips"
- os: "oracle 9.4"
buildConfig: "offline"
supershal marked this conversation as resolved.
Show resolved Hide resolved
- os: "oracle 9.4"
buildConfig: "offline-fips"
- os: "flatcar"
buildConfig: "basic"
runs-on:
Expand All @@ -55,7 +59,7 @@ jobs:
with:
username: ${{ secrets.NEXUS_USERNAME }}
password: ${{ secrets.NEXUS_PASSWORD }}

- name: Login to D2iQ's Mirror Registry
uses: docker/login-action@v3
with:
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/vsphere-e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ jobs:
buildConfig: "basic"
supershal marked this conversation as resolved.
Show resolved Hide resolved
- os: "oracle 9.4"
buildConfig: "fips"
- os: "oracle 9.4"
buildConfig: "offline"
- os: "oracle 9.4"
buildConfig: "offline-fips"
- os: "flatcar"
buildConfig: "basic"
runs-on:
Expand All @@ -58,7 +62,7 @@ jobs:
with:
username: ${{ secrets.NEXUS_USERNAME }}
password: ${{ secrets.NEXUS_PASSWORD }}

- name: Login to D2iQ's Mirror Registry
uses: docker/login-action@v3
with:
Expand All @@ -73,7 +77,7 @@ jobs:
uses: webfactory/[email protected]
with:
ssh-private-key: ${{ secrets.SSH_BASTION_KEY_CONTENTS }}

# configure git to access private repo hosting vsphere module mesosphere/vcenter-tools
- name: Configure git to clone private registry from mesosphere org
run: |
Expand Down
5 changes: 3 additions & 2 deletions ansible/roles/containerd/tasks/redhat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@
retries: 5
delay: 6


- name: install tar rpm package
- name: install tar and container-selinux rpm package
yum:
name: "{{ item }}"
state: present
update_cache: true
enablerepo: "{{ 'offline' if offline_mode_enabled else '' }}"
disablerepo: "{{ '*' if offline_mode_enabled else '' }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

register: result
until: result is success
retries: 5
Expand Down
35 changes: 35 additions & 0 deletions bundles/oracle9.4/bundle.sh.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash
set -x
set -euo pipefail

# When yum operates on multiple packages, it does not, by default, return an error if a subset
# of packages is not found. This makes yum return an error.
echo skip_missing_names_on_install=False >> /etc/yum.conf

# Install required tools for repository creation
yum -y install epel-release gettext dnf-utils createrepo modulemd-tools yum-utils
yum clean all

TMP_DIR="$(mktemp -d repodata-XXXX)"
chmod 777 -R "${TMP_DIR}"
cp packages.txt "${TMP_DIR}"
pushd "${TMP_DIR}"
#shellcheck disable=SC2046
repoquery --archlist=x86_64,noarch --resolve --requires --recursive $(< packages.txt) | xargs -r yumdownloader --archlist=x86_64,noarch -x \*i686
#shellcheck disable=SC2046
yumdownloader --archlist=x86_64,noarch -x \*i686 $(< packages.txt)
rm packages.txt

createrepo -v .
repo2module . --module-name offline
createrepo_mod .

# Package the repository into a tarball with proper permissions
#shellcheck disable=SC1083,SC2035
tar -czf {{ .OutputDirectory }}/{{ .KubernetesVersion }}_oraclelinux_9.4_x86_64{{ .FipsSuffix }}.tar.gz *
#shellcheck disable=SC1083,SC2035
chmod 777 {{ .OutputDirectory }}/{{ .KubernetesVersion }}_oraclelinux_9.4_x86_64{{ .FipsSuffix }}.tar.gz

# Return to the original directory and clean up
popd
rm -rf "${TMP_DIR}"
33 changes: 33 additions & 0 deletions bundles/oracle9.4/packages.txt.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
audit
ca-certificates
conntrack-tools
chrony
open-vm-tools
python3-pip
socat
sysstat
dnf-utils
yum-utils
yum-plugin-versionlock
NetworkManager
cloud-init
cloud-utils-growpart
container-selinux
{{ if .FetchKubernetesRPMs -}}
kubectl-{{ .KubernetesVersion }}-0
kubelet-{{ .KubernetesVersion }}-0
kubeadm-{{ .KubernetesVersion }}-0
cri-tools
{{- end }}
conntrack
ebtables
ethtool
iproute
iptables
socat
gcc
libseccomp
nfs-utils
sssd-kcm
iptables-libs
libnftnl
5 changes: 5 additions & 0 deletions bundles/oracle9.4/repo-templates/kubernetes.repo.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[kubernetes]
name=Konvoy Kubernetes package repository
baseurl=https://packages.d2iq.com/konvoy/stable/linux/repos/el/kubernetes-v{{ .KubernetesVersion }}-{{ .RepoSuffix }}/x86_64
gpgcheck=1
gpgkey=https://packages.d2iq.com/konvoy/stable/linux/repos/d2iq-sign-authority-gpg-public-key
1 change: 0 additions & 1 deletion bundles/redhat8.6/packages.txt.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ sysstat
yum-utils
yum-plugin-versionlock
NetworkManager
yum-utils
supershal marked this conversation as resolved.
Show resolved Hide resolved
cloud-init
cloud-utils-growpart
container-selinux
Expand Down
1 change: 0 additions & 1 deletion bundles/redhat8.8/packages.txt.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ sysstat
yum-utils
yum-plugin-versionlock
NetworkManager
yum-utils
cloud-init
cloud-utils-growpart
container-selinux
Expand Down
1 change: 0 additions & 1 deletion bundles/rocky9.1/packages.txt.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ sysstat
yum-utils
yum-plugin-versionlock
NetworkManager
yum-utils
cloud-init
cloud-utils-growpart
container-selinux
Expand Down
4 changes: 4 additions & 0 deletions cmd/konvoy-image-wrapper/cmd/create-package-bundle.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ var osToConfig = map[string]OSConfig{
configDir: "bundles/ubuntu20.04",
containerImage: "docker.io/library/ubuntu:20.04",
},
"oracle-9.4": {
configDir: "bundles/oracle9.4",
containerImage: "docker.io/library/oraclelinux:9",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the minor version container images have not been released yet on any registry(ref) so using this image which currently points to 9.4 only -

[root@8b53e13be278 generated]# cat /etc/oracle-release
Oracle Linux Server release 9.4

[root@8b53e13be278 generated]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.4 (Plow)

[root@8b53e13be278 generated]# cat /etc/os-release 
NAME="Oracle Linux Server"
VERSION="9.4"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="9.4"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Oracle Linux Server 9.4"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:9:4:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://github.com/oracle/oracle-linux"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 9"
ORACLE_BUGZILLA_PRODUCT_VERSION=9.4
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=9.4

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can update the image tag once 9.4 version is released on their official registry/repo -

  1. https://container-registry.oracle.com/ords/f?p=113:4:10433988665656:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1,1,Oracle%20Linux,Oracle%20Linux,1,0&cs=3GLgmu4zPDtIjZjF1cXhfpBMVzRl0Sefp1HfIY1Aw88xKXzFyVBlZp58aVGnejQ6Yin9zyBXV3AgvO8P-JhWwgw
  2. https://github.com/oracle/container-images/pkgs/container/oraclelinux

Alternatively we can peg image tag to specific SHA but I'm skeptical about reliability of SHA tags to be generally available all time -
Screenshot 2024-10-03 at 4 16 07 PM

},
}

func getKubernetesVerisonFromAnsible() (string, error) {
Expand Down
1 change: 1 addition & 0 deletions cmd/konvoy-image/cmd/create-package-bundle.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ var (
"rocky-9.1",
"ubuntu-18.04",
"ubuntu-20.04",
"oracle-9.4",
}
)

Expand Down
2 changes: 1 addition & 1 deletion docs/cli/konvoy-image_create-package-bundle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ create-package-bundle --os redhat-8.4 --output-directory=artifacts
--fips If the package bundle should include fips packages.
-h, --help help for create-package-bundle
--kubernetes-version string The version of kubernetes to download packages for.
--os string The target OS you wish to create a package bundle for. Must be one of [centos-7.9 redhat-7.9 redhat-8.4 redhat-8.6 redhat-8.8 rocky-9.1 ubuntu-18.04 ubuntu-20.04]
--os string The target OS you wish to create a package bundle for. Must be one of [centos-7.9 redhat-7.9 redhat-8.4 redhat-8.6 redhat-8.8 rocky-9.1 ubuntu-18.04 ubuntu-20.04 oracle-9.4]
--output-directory string The directory to place the bundle in. (default "artifacts")
```

Expand Down
5 changes: 4 additions & 1 deletion magefile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -447,12 +447,15 @@ func fetchPipPackages(downloadDir string) error {

func fetchContainerd(osName, downloadDir, containerdVersion string, fips bool) error {
osInfo := strings.Split(osName, " ")
osDist := osInfo[0]
// TODO: improve this
osMajorMinor := strings.Split(osInfo[1], ".")
osMajor := osMajorMinor[0]
osMinor := osMajorMinor[1]

osDist := osInfo[0]
osDist = strings.Replace(osDist, "redhat", "rhel", 1)
osDist = strings.Replace(osDist, "oracle", "ol", 1)

containerdFile := fmt.Sprintf("containerd-%s-d2iq.1-%s-%s.%s-x86_64", containerdVersion, osDist, osMajor, osMinor)
if fips {
containerdFile += "_fips"
Expand Down
Loading