Running behind Foreman/orcharhino/Satellite

[Fobhep]

I am using ansible-builder on a server that is air-gapped ie it is behind a Foreman/orcharhino/Satellite installation and thus has custom repositories.

Currently my installation is failing, since I can't manage to import the custom GPG key into the container.
This is somewhat connected to #195 and #293

Is there any other workarounds for now except creating a custom base container with said GPG key or modifying the created Dockerfile and building manually?

edit: added log

ansible-builder build  --tag test-ee  -v 3
Ansible Builder is building your execution environment image, "test-ee".
File context/_build/requirements.yml will be created.
File context/_build/bindep.txt will be created.
Rewriting Containerfile to capture collection requirements
Running command:
  podman build -f context/Containerfile -t test-ee context
[1/3] STEP 1/7: FROM registry.redhat.io/ansible-automation-platform-21/ee-minimal-rhel8:latest AS galaxy
[1/3] STEP 2/7: ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=
--> Using cache 4babeebf962f20c0ffaeb72eafb92e836b9834c4e064f532c2ebabba78d6fc75
--> 4babeebf962
[1/3] STEP 3/7: USER root
--> Using cache d528a7e7da3ec33159c76dd0202d3970f0d8b58cbe8ca57115838c11156001e9
--> d528a7e7da3
[1/3] STEP 4/7: ADD _build /build
--> 70ad182cfa9
[1/3] STEP 5/7: WORKDIR /build
--> c2d2e35dc16
[1/3] STEP 6/7: RUN ansible-galaxy role install -r requirements.yml --roles-path /usr/share/ansible/roles
Skipping install, no requirements found
--> 18c8284398e
[1/3] STEP 7/7: RUN ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path /usr/share/ansible/collections
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://galaxy.ansible.com/download/theforeman-foreman-3.1.0.tar.gz to /home/runner/.ansible/tmp/ansible-local-1f4fgnim_/tmp8_opl4gx/theforeman-foreman-3.1.0-s0rk78a0
Installing 'theforeman.foreman:3.1.0' to '/usr/share/ansible/collections/ansible_collections/theforeman/foreman'
theforeman.foreman:3.1.0 was installed successfully
--> 372a21b887e
[2/3] STEP 1/5: FROM registry.redhat.io/ansible-automation-platform-21/ansible-builder-rhel8:latest AS builder
[2/3] STEP 2/5: COPY --from=galaxy /usr/share/ansible /usr/share/ansible
--> Using cache 9c4005a5fa7db4e4b0e1fe154eebe432c88e756846b22c14f4daf6c6a5bdd818
--> 9c4005a5fa7
[2/3] STEP 3/5: ADD _build/bindep.txt bindep.txt
--> 990212b05e3
[2/3] STEP 4/5: RUN ansible-builder introspect --sanitize --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
# Sanitized dependencies for /usr/share/ansible/collections
---
python:
- 'requests>=2.4.2  # from collection theforeman.foreman'
- 'ipaddress  # from collection theforeman.foreman'
system:
- 'python3-rpm [(platform:redhat platform:base-py3)]  # from collection theforeman.foreman'
- 'rpm-python [(platform:redhat platform:base-py2)]  # from collection theforeman.foreman'
- 'libxml2-2.9.7-11.el8.x86_64 [platform:rpm ]  # from collection user'

Creating parent directory for /tmp/src/requirements.txt
--> 6197cc2d29e
[2/3] STEP 5/5: RUN assemble
++ source /etc/os-release
+++ NAME='Red Hat Enterprise Linux'
+++ VERSION='8.5 (Ootpa)'
+++ ID=rhel
+++ ID_LIKE=fedora
+++ VERSION_ID=8.5
+++ PLATFORM_ID=platform:el8
+++ PRETTY_NAME='Red Hat Enterprise Linux 8.5 (Ootpa)'
+++ ANSI_COLOR='0;31'
+++ CPE_NAME=cpe:/o:redhat:enterprise_linux:8::baseos
+++ HOME_URL=https://www.redhat.com/
+++ DOCUMENTATION_URL=https://access.redhat.com/documentation/red_hat_enterprise_linux/8/
+++ BUG_REPORT_URL=https://bugzilla.redhat.com/
+++ REDHAT_BUGZILLA_PRODUCT='Red Hat Enterprise Linux 8'
+++ REDHAT_BUGZILLA_PRODUCT_VERSION=8.5
+++ REDHAT_SUPPORT_PRODUCT='Red Hat Enterprise Linux'
+++ REDHAT_SUPPORT_PRODUCT_VERSION=8.5
++ echo rhel
+ RELEASE=rhel
+ PKGMGR=
+ PKGMGR_OPTS=
+ '[' -z ']'
+ PKGMGR=/usr/bin/dnf
+ '[' -f /usr/bin/microdnf ']'
+ PKGMGR=/usr/bin/microdnf
+ '[' -z ']'
+ PKGMGR_OPTS='--nodocs --setopt install_weak_deps=0'
+ mkdir -p /output/bindep
+ mkdir -p /output/wheels
+ mkdir -p /tmp/src
+ cd /tmp/src
+ /usr/bin/microdnf update -y
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Downloading metadata...
Package                                     Repository                         Size
Upgrading:                                                                         
 libxml2-2.9.7-11.el8.x86_64                ATIX_RHEL_Client_RHEL_Client_8 711.4 kB
   replacing libxml2-2.9.7-9.el8_4.2.x86_64                                        
Transaction Summary:
 Installing:        0 packages
 Reinstalling:      0 packages
 Upgrading:         1 packages
 Obsoleting:        0 packages
 Removing:          0 packages
 Downgrading:       0 packages
Downloading packages...
error: package libxml2-2.9.7-11.el8.x86_64 cannot be verified and repo ATIX_RHEL_Client_RHEL_Client_8 is GPG enabled: /var/cache/yum/metadata/ATIX_RHEL_Client_RHEL_Client_8-8-x86_64/packages/libxml2-2.9.7-11.el8.x86_64.rpm could not be verified.
/var/cache/yum/metadata/ATIX_RHEL_Client_RHEL_Client_8-8-x86_64/packages/libxml2-2.9.7-11.el8.x86_64.rpm:  digest:  SIGNATURE:  NOT OK
[3/3] STEP 1/11: FROM registry.redhat.io/ansible-automation-platform-21/ee-minimal-rhel8:latest
Error: error building at STEP "RUN assemble": error while running runtime: exit status 1

An error occured (rc=125), see output line(s) above for details.

[milanzelenka]

Hi @Fobhep, please where did you specify custom (foreman/satellite) repository url for "/usr/bin/microdnf update -y" command in assemble?

[Fobhep]

@milanzelenka I did not. My understanding is that ansible-builder uses by default those settings on EL servers:

ARG EE_BASE_IMAGE=registry.redhat.io/ansible-automation-platform-21/ee-minimal-rhel8:latest
ARG EE_BUILDER_IMAGE=registry.redhat.io/ansible-automation-platform-21/ansible-builder-rhel8:latest

And I think that within those images the repo list of the server is being mirrored into the actual container.
Maybe somebody can confirm or deny that?

[milanzelenka]

Thanks. It's interesting. I my case it uses default public ubi.redhat.com repository, which time-outs because of no internet access... :-(

error: cannot update repo 'ubi-8-baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried; Last error: Curl error (28): Timeout was reached for https://cdn-ubi.redhat.com/content/public/ubi/dist/ubi8/8/x86_64/baseos/os/repodata/repomd.xml [Connection timed out after 30001 milliseconds]

[Fobhep]

check out your Containerfile in the build directory - what is set for you?
something like this?

ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:latest
ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest

You can always try and prebuild your BASE/BUILDER image by hand in a way that it uses your airgapped repositories and ships corresponding certificates inside that base image

[milanzelenka]

Yes, we are using local registry in private automation hub... It's look like that we need to rebuild ansible-builder-rhel8 image with custom repository lists...

ARG EE_BASE_IMAGE=aah.XXX.cz/ansible-automation-platform-21/ee-minimal-rhel8
ARG EE_BUILDER_IMAGE=aah.XXX.cz/ansible-automation-platform-21/ansible-builder-rhel8

[Rikbruggink]

The default way for a ubi image to react with a system connected to satellite/foreman is to use the system repos if applicable but only if the default ubi.repo is not there. For my set-up i am running this to have the final image to use ansible repo's

additional_build_steps:
prepend:
- RUN rm -f /etc/yum.repos.d/ubi.repo

this doenst work for build deps like galaxy. i prepend them there manually. This is something i need to open with the downstream channels.

[ryanmerolle]

I have this same issue.

I added my internal yum repos into both my base ansible-builder and ansible-runner images. Only those yum repos show up in /etc/yum.repos.d/ and in yum repolist

When assemble runs I get a few pertinent log messages related to the build being air gaped

Repository appstream is listed more than once in the configuration
Repository baseos is listed more than once in the configuration
Repository extras is listed more than once in the configuration
Repository powertools is listed more than once in the configuration
CentOS Stream 8 - AppStream                     0.0  B/s |   0  B     00:01    
Errors during downloading metadata for repository 'appstream':
  - Curl error (56): Failure when receiving data from the peer for http://mirrorlist.centos.org/?release=8-stream&arch=x86_64&repo=AppStream&infra=stock [Recv failure: Connection reset by peer]
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: Curl error (56): Failure when receiving data from the peer for http://mirrorlist.centos.org/?release=8-stream&arch=x86_64&repo=AppStream&infra=stock [Recv failure: Connection reset by peer]

When I run assemble with the same base images on a non-air gaped system, the build completes, but I then see assemble added a number of new yum repos named CentOS-Stream which I did not configure or see before I ran assemble.

[sabre1041]

Experiencing same issue as well. There are no current methods for injection points to customize the behavior of the builder image in a similar fashion as the base