Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] hauler unable to parse complex file names #175

Closed
zackbradys opened this issue Feb 14, 2024 · 7 comments · Fixed by #178
Closed

[BUG] hauler unable to parse complex file names #175

zackbradys opened this issue Feb 14, 2024 · 7 comments · Fixed by #178
Assignees
@amartin120
Labels
bug Something isn't working size/M Denotes an issue/PR requiring a relatively moderate amount of work
Milestone
Hauler v1.0.0

Comments

@zackbradys
Copy link
Member

zackbradys commented Feb 14, 2024 •
edited
Loading

Environmental Info:

  • N/A

Hauler Version:

[root@ip-172-31-34-117 rocky]# hauler version
 __    __       ___       __    __   __       _______ .______
|  |  |  |     /   \     |  |  |  | |  |     |   ____||   _  \
|  |__|  |    /  ^  \    |  |  |  | |  |     |  |__   |  |_)  |
|   __   |   /  /_\  \   |  |  |  | |  |     |   __|  |      /
|  |  |  |  /  _____  \  |  `--'  | |  `----.|  |____ |  |\  \----.
|__|  |__| /__/     \__\  \______/  |_______||_______|| _| `._____|
hauler: Airgap Swiss Army Knife

GitVersion:    0.4.4
GitCommit:     3cf4afe
GitTreeState:  clean
BuildDate:     2024-02-13T00:39:31Z
GoVersion:     go1.21.6
Compiler:      gc
Platform:      linux/amd64

System CPU architecture, OS, and Version:

Linux ip-172-31-34-117.ec2.internal 5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov 8 17:36:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug:

  • When using hauler with complex file names, or even with random file names that are not seemly complex, such as linux packages and/or rpms, it will fail to add to the store with hauler store add file or hauler store sync.
  • Specifically, in the below hauler manifest, hauler "could not parse reference" for any name with perl or libstdc

Steps To Reproduce:

  • hauler store sync --store packages --files rancher-airgap-packages.yaml

Example Errors:

  • 2:26AM ERR could not parse reference: perl-Time-Local-1.300-7.el9.noarch.rpm
  • 2:28AM ERR could not parse reference: perl-MIME-Base64-3.16-4.el9.x86_64.rpm
  • 2:31AM ERR could not parse reference: perl-TermReadKey-2.38-11.el9.x86_64.rpm
  • 2:32AM ERR could not parse reference: libstdc++-11.4.1-2.1.el9.x86_64.rpm

Expected behavior:

  • All files to be able to be added to hauler

Actual behavior:

  • Not all files are able to be added to hauler

Additional context / logs:

  • rancher-airgap-packages.yaml:
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
  name: rancher-airgap-packages
spec:
  files:
    - path: /opt/rancher/hauler/repos
    - path: /opt/rancher/hauler/repos/dbus-common-1.12.20-8.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libxcrypt-4.4.18-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python-unversioned-command-3.9.18-1.el9_3.1.noarch.rpm
    - path: /opt/rancher/hauler/repos/libevent-2.1.12-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libxcrypt-4.4.18-3.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/basesystem-11-13.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/container-selinux-2.221.0-1.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/setup-2.13.7-9.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/python3-distro-1.5.0-7.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/selinux-policy-targeted-38.1.23-1.el9_3.2.noarch.rpm
    - path: /opt/rancher/hauler/repos/selinux-policy-38.1.23-1.el9_3.2.noarch.rpm
    - path: /opt/rancher/hauler/repos/nettle-3.8-3.el9_0.x86_64.rpm
    - path: /opt/rancher/hauler/repos/pcre-8.44-3.el9.3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/json-c-0.14-11.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libcbor-0.7.0-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libacl-2.3.1-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/glibc-gconv-extra-2.34-83.el9.7.x86_64.rpm
    - path: /opt/rancher/hauler/repos/glibc-common-2.34-83.el9.7.x86_64.rpm
    - path: /opt/rancher/hauler/repos/glibc-2.34-83.el9.7.x86_64.rpm
    - path: /opt/rancher/hauler/repos/crypto-policies-20230731-1.git94f0e2c.el9_3.1.noarch.rpm
    - path: /opt/rancher/hauler/repos/zip-3.0-35.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/glibc-gconv-extra-2.34-83.el9.7.i686.rpm
    - path: /opt/rancher/hauler/repos/glibc-2.34-83.el9.7.i686.rpm
    - path: /opt/rancher/hauler/repos/acl-2.3.1-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libyaml-0.2.5-7.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libmount-2.37.4-15.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libblkid-2.37.4-15.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/perl-Time-Local-1.300-7.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/util-linux-core-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/util-linux-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/expat-2.5.0-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libbrotli-1.0.9-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libuuid-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/bzip2-libs-1.0.8-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libsmartcols-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libmount-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libfdisk-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libblkid-2.37.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/groff-base-1.22.4-10.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/openssh-clients-8.7p1-34.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/openssh-8.7p1-34.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/glib2-2.68.4-11.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/file-libs-5.39-14.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libref_array-0.1.5-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/audit-libs-3.0.7-104.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libpath_utils-0.2.1-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libini_config-1.3.1-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libcollection-0.7.0-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libbasicobjects-0.1.1-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/alternatives-1.24-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libeconf-0.4.1-3.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/lua-libs-5.4.4-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/lz4-libs-1.9.3-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-libsemanage-3.5-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libidn2-2.3.0-7.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-libselinux-3.5-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libcap-ng-0.8.2-7.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/device-mapper-libs-1.02.195-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/iptables-nft-1.8.8-6.el9_1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/iptables-libs-1.8.8-6.el9_1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/device-mapper-1.02.195-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-MIME-Base64-3.16-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libarchive-3.5.3-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libverto-libev-0.3.2-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libverto-0.3.2-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libxml2-2.9.13-5.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libdb-5.3.28-53.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-setools-4.4.3-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/policycoreutils-3.5-3.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnfnetlink-1.0.1-21.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/isns-utils-libs-0.101-4.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/pcre2-10.40-2.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libibverbs-46.0-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/xz-libs-5.2.5-8.el9_0.i686.rpm
    - path: /opt/rancher/hauler/repos/popt-1.18-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-pip-wheel-21.2.3-7.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/rpm-plugin-selinux-4.16.1.3-27.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/rpm-libs-4.16.1.3-27.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/rpm-4.16.1.3-27.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/publicsuffix-list-dafsa-20210518-3.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/quota-4.06-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/quota-nls-4.06-6.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/openssl-libs-3.0.7-25.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/openssl-3.0.7-25.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/pcre2-syntax-10.40-2.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/pcre2-10.40-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/dbus-broker-28-7.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libpcap-1.10.0-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnftnl-1.2.2-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/ncurses-base-6.2-10.20210508.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/rocky-repos-9.3-1.2.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/rocky-release-9.3-1.2.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/rocky-gpg-keys-9.3-1.2.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libtirpc-1.3.3-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/p11-kit-0.24.1-2.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/p11-kit-trust-0.24.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/p11-kit-0.24.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-pyyaml-5.4.1-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/systemd-libs-252-18.el9.0.1.rocky.i686.rpm
    - path: /opt/rancher/hauler/repos/python3-setuptools-wheel-53.0.0-12.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/python3-setuptools-53.0.0-12.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/systemd-rpm-macros-252-18.el9.0.1.rocky.noarch.rpm
    - path: /opt/rancher/hauler/repos/systemd-pam-252-18.el9.0.1.rocky.x86_64.rpm
    - path: /opt/rancher/hauler/repos/oniguruma-6.9.6-1.el9.5.x86_64.rpm
    - path: /opt/rancher/hauler/repos/systemd-libs-252-18.el9.0.1.rocky.x86_64.rpm
    - path: /opt/rancher/hauler/repos/oniguruma-6.9.6-1.el9.5.i686.rpm
    - path: /opt/rancher/hauler/repos/systemd-252-18.el9.0.1.rocky.x86_64.rpm
    - path: /opt/rancher/hauler/repos/bash-5.1.8-6.el9_1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/tzdata-2024a-1.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libutempter-1.2.1-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/createrepo_c-libs-0.20.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/createrepo_c-0.20.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-TermReadKey-2.38-11.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/grep-3.6-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/gssproxy-0.8.4-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libfido2-1.13.0-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnetfilter_conntrack-1.0.9-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libcurl-7.76.1-26.el9_3.2.0.1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/curl-7.76.1-26.el9_3.2.0.1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/rpcbind-1.2.6-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/openldap-2.6.3-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/cryptsetup-2.6.0-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/cryptsetup-libs-2.6.0-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/shadow-utils-4.9-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libssh-config-0.10.4-11.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libssh-0.10.4-11.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Net-SSLeay-1.92-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Carp-1.50-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Text-ParseWords-3.30-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/sed-4.8-9.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/cracklib-dicts-2.9.6-27.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/cracklib-2.9.6-27.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Exporter-5.74-461.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/gzip-1.12-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnftnl-1.2.2-1.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/kmod-libs-28-9.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/kmod-28-9.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-podlators-4.14-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Term-Cap-1.17-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libxcrypt-compat-4.4.18-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Term-ANSIColor-5.01-461.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/isns-utils-libs-0.101-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnfnetlink-1.0.1-21.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-HTTP-Tiny-0.076-461.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libmnl-1.0.4-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Digest-1.19-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libev-4.33-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Pod-Simple-3.42-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/ncurses-libs-6.2-10.20210508.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/gawk-all-langpacks-5.1.0-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/ncurses-6.2-10.20210508.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-URI-5.09-3.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-constant-1.33-461.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/tree-1.8.0-10.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Encode-3.08-462.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libgcc-11.4.1-2.1.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/less-590-2.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/nfs-utils-2.5.4-20.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnfsidmap-2.5.4-20.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libnghttp2-1.43.0-5.el9_3.1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/readline-8.1-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-policycoreutils-3.5-3.el9_3.noarch.rpm
    - path: /opt/rancher/hauler/repos/policycoreutils-python-utils-3.5-3.el9_3.noarch.rpm
    - path: /opt/rancher/hauler/repos/sqlite-libs-3.34.1-7.el9_3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Socket-2.031-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/krb5-libs-1.21.1-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/xz-libs-5.2.5-8.el9_0.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libsigsegv-2.13-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/cyrus-sasl-lib-2.1.27-21.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/checkpolicy-3.5-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Git-2.39.3-1.el9_2.noarch.rpm
    - path: /opt/rancher/hauler/repos/keyutils-libs-1.6.3-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/keyutils-1.6.3-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/gawk-5.1.0-6.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Data-Dumper-2.174-462.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/pam-1.5.1-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/git-core-doc-2.39.3-1.el9_2.noarch.rpm
    - path: /opt/rancher/hauler/repos/filesystem-3.16-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/gdbm-libs-1.19-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libgcrypt-1.10.0-10.el9_2.i686.rpm
    - path: /opt/rancher/hauler/repos/unzip-6.0-56.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-File-Path-2.18-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Pod-Perldoc-3.28.01-461.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libzstd-1.5.1-2.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/zstd-1.5.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libzstd-1.5.1-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libunistring-0.9.10-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/zlib-1.2.11-40.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/jq-1.6-15.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/jq-1.6-15.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libstdc++-11.4.1-2.1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libgcc-11.4.1-2.1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Scalar-List-Utils-1.56-461.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libmnl-1.0.4-15.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/glibc-minimal-langpack-2.34-83.el9.7.x86_64.rpm
    - path: /opt/rancher/hauler/repos/lz4-libs-1.9.3-5.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/openssl-libs-3.0.7-25.el9_3.i686.rpm
    - path: /opt/rancher/hauler/repos/perl-Getopt-Long-2.52-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libnetfilter_conntrack-1.0.9-1.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/perl-IO-Socket-SSL-2.073-1.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-libnet-3.13-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/zlib-1.2.11-40.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/kmod-libs-28-9.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/coreutils-common-8.32-34.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/coreutils-8.32-34.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-libs-3.9.18-1.el9_3.1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-3.9.18-1.el9_3.1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/diffutils-3.7-12.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/gnutls-3.7.6-23.el9_3.3.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-mro-1.23-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Pod-Usage-2.01-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-libs-5.32.1-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-lib-0.65-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-interpreter-5.32.1-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-parent-0.238-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Storable-3.21-460.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-POSIX-1.94-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-NDBM_File-1.15-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-IO-1.43-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-PathTools-3.78-461.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Fcntl-1.13-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Errno-1.30-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-DynaLoader-1.47-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libpwquality-1.4.4-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libattr-2.5.1-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Mozilla-CA-20200520-6.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-B-1.80-480.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libffi-3.4.2-8.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libffi-3.4.2-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-vars-1.05-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-subs-1.03-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/gmp-6.2.0-13.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Text-Tabs+Wrap-2013.0523-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-overloading-0.02-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libgcrypt-1.10.0-10.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Digest-MD5-2.58-4.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-overload-1.31-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libcap-2.48-9.el9_2.i686.rpm
    - path: /opt/rancher/hauler/repos/libedit-3.1-38.20210216cvs.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libseccomp-2.5.2-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-if-0.60.800-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libcap-2.48-9.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-base-2.27-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libsepol-3.5-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libsepol-3.5-1.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/ca-certificates-2023.2.60_v7.0.306-90.1.el9_2.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Symbol-1.08-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libcom_err-1.46.5-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/e2fsprogs-libs-1.46.5-3.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-SelectSaver-1.02-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Pod-Escapes-1.07-460.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libtasn1-4.16.0-8.el9_1.x86_64.rpm
    - path: /opt/rancher/hauler/repos/python3-audit-3.0.7-104.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/git-2.39.3-1.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-IPC-Open3-1.21-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/mpfr-4.1.0-7.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/git-core-2.39.3-1.el9_2.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Getopt-Std-1.12-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-FileHandle-2.03-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libnl3-3.7.0-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-File-stat-1.09-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libsemanage-3.5-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libselinux-3.5-1.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libselinux-utils-3.5-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libselinux-3.5-1.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/libgpg-error-1.42-5.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/libgpg-error-1.42-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-Error-0.17029-7.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libpsl-0.21.1-5.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-File-Find-1.37-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-File-Basename-2.85-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/emacs-filesystem-27.2-9.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/iscsi-initiator-utils-6.2.1.4-3.git2a8f9d8.el9.i686.rpm
    - path: /opt/rancher/hauler/repos/iscsi-initiator-utils-iscsiuio-6.2.1.4-3.git2a8f9d8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/iscsi-initiator-utils-6.2.1.4-3.git2a8f9d8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/perl-IO-Socket-IP-0.41-5.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-Class-Struct-0.66-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-AutoLoader-5.74-480.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/perl-File-Temp-0.231.100-4.el9.noarch.rpm
    - path: /opt/rancher/hauler/repos/libmodulemd-2.13.0-2.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/dbus-1.12.20-8.el9.x86_64.rpm
    - path: /opt/rancher/hauler/repos/repodata
    - path: /opt/rancher/hauler/repos/repodata/7328895d5814edd3eed11249c4d8d2e252c40a6049cd7b97f963adba6e6ec5b1-primary.xml.gz
    - path: /opt/rancher/hauler/repos/repodata/e9b046e7434b3a3affe2bef268c520edfea953e78bcf8e05dcca9803fdf55ef2-filelists.xml.gz
    - path: /opt/rancher/hauler/repos/repodata/447a1e082b5f6d838af2d9f197c7f43d0ac8bcfb4fcb8a4bbfd9ac802199cfad-other.xml.gz
    - path: /opt/rancher/hauler/repos/repodata/4ef0523911fdfde1c96691d54dc8d6e94a9f8670b03133d27af2e992da6b1c09-primary.sqlite.bz2
    - path: /opt/rancher/hauler/repos/repodata/25047b86727ce109097e6a6c7dcc12e289c5cf7cb396b118a15d5c7529bfdd1e-filelists.sqlite.bz2
    - path: /opt/rancher/hauler/repos/repodata/53c42c3349f764a655e57829cdd9c7f5edd519c0c2f7ab64eccf40524fac2502-other.sqlite.bz2
    - path: /opt/rancher/hauler/repos/repodata/repomd.xml
    - path: /opt/rancher/hauler/repos/package-list.txt
@zackbradys zackbradys added the bug Something isn't working label Feb 14, 2024
@github-project-automation github-project-automation bot moved this to Pending Review in Hauler Feb 14, 2024
@zackbradys
Copy link
Member Author 

zackbradys@zacks-mbp Downloads % hauler store add file https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libstdc%2B%2B-11.4.1-2.1.el9.x86_64.rpm
Error: could not parse reference: libstdc%!B(MISSING)%!B(MISSING)-11.4.1-2.1.el9.x86_64.rpm
Usage:
  hauler store add file [flags]

Flags:
  -h, --help          help for file
  -n, --name string   (Optional) Name to assign to file in store

Global Flags:
      --cache string       Location of where to store cache data (defaults to $XDG_CACHE_DIR/hauler)
  -l, --log-level string    (default "info")
  -s, --store string       Location to create store at (default "store")

10:37PM ERR could not parse reference: libstdc%!B(MISSING)%!B(MISSING)-11.4.1-2.1.el9.x86_64.rpm

@clemenko
Copy link
Contributor

Hauler is not able to parse "%2B". Converting it to "+" does not work either.

@zackbradys
Copy link
Member Author

@clemenko thanks for the confirmation. the browser automatically generated the link with the %2B instead of +, but it's other files as well...

zackbradys@zacks-mbp Downloads % hauler store add file https://download.rockylinux.org/pub/rocky/9/AppStream/x86_64/os/Packages/p/perl-TermReadKey-2.38-11.el9.x86_64.rpm
Error: could not parse reference: perl-TermReadKey-2.38-11.el9.x86_64.rpm
Usage:
  hauler store add file [flags]

Flags:
  -h, --help          help for file
  -n, --name string   (Optional) Name to assign to file in store

Global Flags:
      --cache string       Location of where to store cache data (defaults to $XDG_CACHE_DIR/hauler)
  -l, --log-level string    (default "info")
  -s, --store string       Location to create store at (default "store")

8:19AM ERR could not parse reference: perl-TermReadKey-2.38-11.el9.x86_64.rpm

@amartin120 amartin120 self-assigned this Feb 15, 2024
@amartin120
Copy link
Contributor

Good catch. Names with "+" and upper case letters don't convert 1:1 into image refs. Let me get Hauler "files" doing some character conversions and that should fix this up.

@amartin120 amartin120 mentioned this issue Feb 16, 2024
Merged
3 tasks
@amartin120
Copy link
Contributor

@zackbradys #178

@zackbradys zackbradys added this to the Hauler v1.0.0 milestone Feb 16, 2024
@zackbradys zackbradys linked a pull request Feb 16, 2024 that will close this issue
bug-fix: handle complex file names #178
Merged
3 tasks
@zackbradys zackbradys moved this from Pending Review to Testing and Validation in Hauler Feb 16, 2024
@zackbradys zackbradys added the size/M Denotes an issue/PR requiring a relatively moderate amount of work label Feb 16, 2024
@github-project-automation github-project-automation bot moved this from Testing and Validation to Resolved in Hauler Feb 16, 2024
@zapalotta
Copy link

I sthis really fixed? In my case, the = is not in the file but in the path of the URL:

[root@le-hauler01-t hauler]# hauler version GitVersion: 1.0.1 GitCommit: bb2a8bf GitTreeState: clean BuildDate: 2024-02-27T16:30:03Z GoVersion: go1.21.7 Compiler: gc Platform: linux/amd64
Files with encoded +:

`apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: files
spec:
files:

- name: https://github.com/rancher/rke2-packaging/releases/download/v1.29.2%2Brke2r1.stable.0/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm

`

Result:

`

hauler store sync --store rke2-store --platform linux/amd64 --files manifest_files.yml -l debug

8:38AM DBG running cli command [hauler store sync]
8:38AM DBG using store at /opt/hauler/rke2-store
8:38AM DBG processing content file: 'manifest_files.yml'
8:38AM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
Error: could not parse reference: https://github.com/rancher/rke2-packaging/releases/download/v1.29.2%!b(MISSING)rke2r1.stable.0/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm
Usage:
...
8:38AM ERR could not parse reference: https://github.com/rancher/rke2-packaging/releases/download/v1.29.2%!b(MISSING)rke2r1.stable.0/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm
`

With '+' in url:

`
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: files
spec:
files:

- name: https://github.com/rancher/rke2-packaging/releases/download/v1.29.2+rke2r1.stable.0/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm

`

Result:

`

hauler store sync --store rke2-store --platform linux/amd64 --files manifest_files.yml -l debug

8:39AM DBG running cli command [hauler store sync]
8:39AM DBG using store at /opt/hauler/rke2-store
8:39AM DBG processing content file: 'manifest_files.yml'
8:39AM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
Error: source : no getter type found matching reference
Usage:
...
8:39AM ERR source : no getter type found matching reference

`

Best regards,
Dirk

@amartin120
Copy link
Contributor

amartin120 commented Mar 21, 2024 •
edited
Loading

@zapalotta

I think it's because you're putting the path in the name and not the path.

You only want to use the name attribute if you want to rename the file, and omitting name will default to the original file name that's part of the path.

for example....

apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
  name: rancher-airgap-packages
spec:
  files:
    - path: https://github.com/rancher/rke2-packaging/releases/download/v1.29.2+rke2r1.stable.0/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm

> hauler store sync -f stuff.yaml                                                                                                                                                                                                                                                                              
8:19AM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
8:19AM INF added 'file' to store at [hauler/rke2-common-1.29.2.rke2r1-0.el9.x86_64.rpm:latest], with digest [sha256:6b822e3d3d4cad349b2835f74b14aada0e53076f89813920fcb1b8fb0c69bc92]

@github-project-automation github-project-automation bot moved this to Pending Review in Hauler Jul 17, 2024
@zackbradys zackbradys moved this from Pending Review to Resolved in Hauler Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amartin120 amartin120

Labels
bug Something isn't working size/M Denotes an issue/PR requiring a relatively moderate amount of work
Projects
Hauler
Status: Resolved
Milestone
Hauler v1.0.0
Development

Successfully merging a pull request may close this issue.

bug-fix: handle complex file names amartin120/hauler
4 participants
@clemenko @zapalotta @zackbradys @amartin120