Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug-fix: handle complex file names #178

Merged
merged 1 commit into from
Feb 16, 2024

Conversation

amartin120
Copy link
Contributor

Please check below, if the PR fulfills these requirements:

  • The commit message follows the guidelines.
  • Tests for the changes have been added (for bug fixes / features).
  • Docs have been added / updated (for bug fixes / features).

What kind of change does this PR introduce?

What is the current behavior?

What is the new behavior (if this is a feature change)?

  • hauler will unescape url file paths when creating the reference name.
  • hauler will make sure that file names are converted to lower case when creating the reference name.
  • hauler will replace "+" with "-" when creating the reference name.
  • hauler will keep the original file name intact despite the above conversions for image reference name.

Does this PR introduce a breaking change?

  • N/A

Other information:

My testing:

apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
  name: rancher-airgap-packages
spec:
  files:
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libstdc%2B%2B-11.4.1-2.1.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/p/perl-TermReadKey-2.38-11.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/p/perl-Text-ParseWords-3.30-460.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/d/dbus-common-1.12.20-8.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libxcrypt-4.4.18-3.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/p/python-unversioned-command-3.9.18-1.el9_3.1.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libevent-2.1.12-6.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libxcrypt-4.4.18-3.el9.i686.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/b/basesystem-11-13.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/c/container-selinux-2.221.0-1.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/s/setup-2.13.7-9.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/p/python3-distro-1.5.0-7.el9.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/s/selinux-policy-targeted-38.1.23-1.el9_3.2.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/s/selinux-policy-38.1.23-1.el9_3.2.noarch.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/n/nettle-3.8-3.el9_0.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/p/pcre-8.44-3.el9.3.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/j/json-c-0.14-11.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libcbor-0.7.0-5.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/l/libacl-2.3.1-3.el9.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/g/glibc-gconv-extra-2.34-83.el9.7.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/g/glibc-common-2.34-83.el9.7.x86_64.rpm
    - path: https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/Packages/g/glibc-2.34-83.el9.7.x86_64.rpm
❯ hauler store sync -f quick-test.yaml                                                                                                                                                                                                                                                        
8:08PM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
8:08PM INF added 'file' to store at [hauler/libstdc---11.4.1-2.1.el9.x86_64.rpm:latest], with digest [sha256:30ac1ad1a0e90c93ebe3dfd5a05f731b4f7f25b6537dba9d9ecf8174cd17e7bc]
8:08PM INF added 'file' to store at [hauler/perl-termreadkey-2.38-11.el9.x86_64.rpm:latest], with digest [sha256:cb95da0a3190dd79cc39aebda359bebdfea1e619671296923d78ed2e875335c4]
8:08PM INF added 'file' to store at [hauler/perl-text-parsewords-3.30-460.el9.noarch.rpm:latest], with digest [sha256:aed280292d2c6e45a3c0cecfd8e0d57b165c0c041affa0712b23470d7fbd965e]
8:08PM INF added 'file' to store at [hauler/dbus-common-1.12.20-8.el9.noarch.rpm:latest], with digest [sha256:8139c8dd8b8dfce069e20e4ec9a42d9989555ac44a609d4abf96decdb5d5024b]
8:08PM INF added 'file' to store at [hauler/libxcrypt-4.4.18-3.el9.x86_64.rpm:latest], with digest [sha256:e55139115dabaed38fa0255681ab67bf0e3099fba40939c97e037b53895fb130]
8:08PM INF added 'file' to store at [hauler/python-unversioned-command-3.9.18-1.el9_3.1.noarch.rpm:latest], with digest [sha256:779d709465be347efd9f5ff60ad20c4d8a613f2ad92cef8998ccac2d8e723d9f]
8:08PM INF added 'file' to store at [hauler/libevent-2.1.12-6.el9.x86_64.rpm:latest], with digest [sha256:1504527b5f8101e3b18fa4709bd52b1e4e0aab1589b3fa7450f13bc626d0f456]
8:08PM INF added 'file' to store at [hauler/libxcrypt-4.4.18-3.el9.i686.rpm:latest], with digest [sha256:8758b6148e1f1cbc42bce8778596bdbdde528380f2edc124a0a77610e20ed5d4]
8:08PM INF added 'file' to store at [hauler/basesystem-11-13.el9.noarch.rpm:latest], with digest [sha256:51d6e3bbae29a480c139b119278a0a143f778a5c0f7b8b75796583dcc46f9e2b]
8:08PM INF added 'file' to store at [hauler/container-selinux-2.221.0-1.el9.noarch.rpm:latest], with digest [sha256:0bc95fe39b032298e4760b4354e4237dda52f4286b4f9f1366e4d2f24322ebdf]
8:08PM INF added 'file' to store at [hauler/setup-2.13.7-9.el9.noarch.rpm:latest], with digest [sha256:c43bde781b9e59f1e509afd60724b41732ea27d542011478ba5fd8bf20275560]
8:08PM INF added 'file' to store at [hauler/python3-distro-1.5.0-7.el9.noarch.rpm:latest], with digest [sha256:45573dacccf0b0142f4e96ff3f4be7a66a6fba367815c3f5e815bbffb05c764f]
8:08PM INF added 'file' to store at [hauler/selinux-policy-targeted-38.1.23-1.el9_3.2.noarch.rpm:latest], with digest [sha256:9c2549c146760aa2f9b110d88f1f1b6bf30fc911fb5d1e23c3ef61a7a78b9b04]
8:08PM INF added 'file' to store at [hauler/selinux-policy-38.1.23-1.el9_3.2.noarch.rpm:latest], with digest [sha256:c25589b15bfb5e0a10d7f69be5de0f61d4305a645a9d3735db5437c67927cc16]
8:08PM INF added 'file' to store at [hauler/nettle-3.8-3.el9_0.x86_64.rpm:latest], with digest [sha256:99c10668ce506ad4562f51af11e6440f69b1ab0a70916811b0839783756f1c03]
8:08PM INF added 'file' to store at [hauler/pcre-8.44-3.el9.3.x86_64.rpm:latest], with digest [sha256:32a2a9e1568c53c733ae4aaa1142a078967eda418248dcb2e2be5f0d510f5e67]
8:08PM INF added 'file' to store at [hauler/json-c-0.14-11.el9.x86_64.rpm:latest], with digest [sha256:8ead98d28b262b217af7c5c8b23f227f163f96f9f99c8de2ead9872dafa3f182]
8:08PM INF added 'file' to store at [hauler/libcbor-0.7.0-5.el9.x86_64.rpm:latest], with digest [sha256:41c26c1f8561719ad0227530dfcf07ad62172b6faca35c94fca1ab37b3b86bef]
8:08PM INF added 'file' to store at [hauler/libacl-2.3.1-3.el9.x86_64.rpm:latest], with digest [sha256:cd3e4be67d89671f7a81e8b4e9d17eb9095f3888b447b6b44fe50cdf75eb1928]
8:08PM INF added 'file' to store at [hauler/glibc-gconv-extra-2.34-83.el9.7.x86_64.rpm:latest], with digest [sha256:f929312a0e1360dd6e957eab115efac7d714c304fe4598ad5ecb92f0861d92dd]
8:08PM INF added 'file' to store at [hauler/glibc-common-2.34-83.el9.7.x86_64.rpm:latest], with digest [sha256:483976c696746e1de5bb17e754f833e2f1744ebdacc8397b0e4efaa8e4be72ba]
8:08PM INF added 'file' to store at [hauler/glibc-2.34-83.el9.7.x86_64.rpm:latest], with digest [sha256:706dfb2c08a5f6415ddea9b9e1cf4169697a5e6f845fdfb8d986867356b2e4cb]
❯ hauler store extract hauler/libstdc---11.4.1-2.1.el9.x86_64.rpm:latest  
❯ ls -la
-rw-r--r--   1 amartin  staff  753710 Feb 15 20:10 libstdc++-11.4.1-2.1.el9.x86_64.rpm

@zackbradys
Copy link
Member

Thanks for tackling this @amartin120. Did you do any testing with local files? Remote files did fail for me, so that's good to see they were fixed, but so did local files.

@zackbradys zackbradys linked an issue Feb 16, 2024 that may be closed by this pull request
@amartin120
Copy link
Contributor Author

amartin120 commented Feb 16, 2024

@zackbradys Yes sir!

❯ hauler store add file libstdc++-11.4.1-2.1.el9.x86_64.rpm                                                                                                                                                                                                                                   
8:19PM INF added 'file' to store at [hauler/libstdc---11.4.1-2.1.el9.x86_64.rpm:latest], with digest [sha256:f5635d341ecaa9587e1cfa94c4e0a3f343b3b0ae83b8c295abf5c228291be652]
❯ hauler store add file perl-TermReadKey-2.38-11.el9.x86_64.rpm                                                                                                                                                                                                                               
8:21PM INF added 'file' to store at [hauler/perl-termreadkey-2.38-11.el9.x86_64.rpm:latest], with digest [sha256:d25259649eaa17ad3802326c538891aab73195d746292a0a5a6eb6116b9d1185]

Copy link
Member

@zackbradys zackbradys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀 🚀

@zackbradys
Copy link
Member

zackbradys commented Feb 16, 2024

If anyone is curious on the use case and how the issue was discovered: https://github.com/zackbradys/rancher-airgap/blob/main/examples/os-packages-example.md

@zackbradys zackbradys added this to the Hauler v1.0.0 milestone Feb 16, 2024
@amartin120 amartin120 merged commit 72734ec into hauler-dev:main Feb 16, 2024
1 check passed
@amartin120 amartin120 deleted the bug-file-name branch February 16, 2024 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] hauler unable to parse complex file names
2 participants