introspect.py - problem parsing requirements.txt - Ansible-Builder-3.0.0

[spikefishjohn]

This is a cross post to a problem adding the collection paloaltonetworks.panos

Warning: failed to parse requirements from paloaltonetworks.panos, error: not enough values to unpack (expected 1, got 0) #447

The core issue seems to be parsing the requirements.txt from the collection.

--> ec10a2c35f7
[3/4] STEP 12/13: RUN $PYCMD /output/scripts/introspect.py introspect --sanitize --user-pip=requirements.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
Warning: failed to parse requirements from paloaltonetworks.panos, error: not enough values to unpack (expected 1, got 0)
Creating parent directory for /tmp/src/requirements.txt
---

requirements.txt from Palo repo

root@debian:~/TestIssue# cat Palo/requirements.txt.orig
pan-os-python==1.8.0 ; python_version >= "3.8" and python_version < "4.0" \
    --hash=sha256:843972122ef4c5bfd17354070e3ada1285e243325f5e7cf7fd969150048becd4 \
    --hash=sha256:da9153fe4a961a43d36016851ab06df45a37ea8612ed36b3b5b528f8394f34f5
pan-python==0.17.0 ; python_version >= "3.8" and python_version < "4.0" \
    --hash=sha256:9c074ea2f69a63996a6fefe8935d60dca61660e14715ac19d257ea9b1c41c6e2 \
    --hash=sha256:f4674e40763c46d5933244b3059a57884e4e28205ef6d0f9ce2dc2013e3db010
xmltodict==0.12.0 ; python_version >= "3.8" and python_version < "4.0" \
    --hash=sha256:50d8c638ed7ecb88d90561beedbf720c9b4e851a9fa6c47ebd64e99d166d8a21 \
    --hash=sha256:8bbcb45cc982f48b2ca8fe7e7827c5d792f217ecf1792626f808bf41c3b86051
root@debian:~/TestIssue#

I tried removing the "\" and joining the lines in question and got the following output.

root@debian:~/TestIssue# cat Palo/requirements.txt
pan-os-python==1.8.0 ; python_version >= "3.8" and python_version < "4.0"  --hash=sha256:843972122ef4c5bfd17354070e3ada1285e243325f5e7cf7fd969150048becd4  --hash=sha256:da9153fe4a961a43d36016851ab06df45a37ea8612ed36b3b5b528f8394f34f5
pan-python==0.17.0 ; python_version >= "3.8" and python_version < "4.0"  --hash=sha256:9c074ea2f69a63996a6fefe8935d60dca61660e14715ac19d257ea9b1c41c6e2  --hash=sha256:f4674e40763c46d5933244b3059a57884e4e28205ef6d0f9ce2dc2013e3db010
xmltodict==0.12.0 ; python_version >= "3.8" and python_version < "4.0"  --hash=sha256:50d8c638ed7ecb88d90561beedbf720c9b4e851a9fa6c47ebd64e99d166d8a21  --hash=sha256:8bbcb45cc982f48b2ca8fe7e7827c5d792f217ecf1792626f808bf41c3b86051
root@debian:~/TestIssue#

root@debian:~/TestIssue# python3 introspect.py introspect --sanitize --user-pip=Palo/requirements.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
Warning: failed to parse requirements from user, error: Parse error at "'--hash=s'": Expected stringEnd
---
python: []
system: []

root@debian:~/TestIssue#

I removed the --hash strings and then it seemed like introspect worked.

root@debian:~/TestIssue# cat Palo/requirements.txt
pan-os-python==1.8.0 ; python_version >= "3.8" and python_version < "4.0"
pan-python==0.17.0 ; python_version >= "3.8" and python_version < "4.0"
xmltodict==0.12.0 ; python_version >= "3.8" and python_version < "4.0"
root@debian:~/TestIssue# python3 introspect.py introspect --sanitize --user-pip=Palo/./requirements.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
---
python:
- 'pan_os_python==1.8.0  # from collection user'
- 'pan_python==0.17.0  # from collection user'
- 'xmltodict==0.12.0  # from collection user'
system: []

root@debian:~/TestIssue#

[spikefishjohn]

FYI ansible-builder came from RH8 repo but I replicated the issue by copying over the files in question to a Debian 11 box. I copied over the site-packages and context/_build/scripts for testing replication.

If this is too much heresy please let me know.

[akira6592]

Warning: failed to parse requirements from user, error: Parse error at "'--hash=s'": Expected stringEnd

The lack of support for --hash seems to be an issue with the requirements-parser that ansible-builder relies on.

issue: madpah/requirements-parser#51
PR: madpah/requirements-parser#67

[Shrews]

Duplicate of issue #319.

Yes, this is due to our use of requirements-parser. There has been some investigation of improving this, but there hasn't been any movement for a while due to other pressing things.

[Shrews]

Closing as duplicate.