Regression of pyVim import fails on macOS because of capital 'V' #487

[agowa]

SUMMARY

importing pyVim on MacOS sometimes fails (M3 Macbook)

ISSUE TYPE

• Bug Report

COMPONENT NAME

community/vmware/plugins/module_utils/vmware.py

ANSIBLE VERSION

ansible [core 2.17.2]
  config file = /Users/user/git/ansible-playbook/ansible.cfg
  configured module search path = ['/Users/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/user/git/ansible-playbook/venv/lib/python3.13/site-packages/ansible
  ansible collection location = /Users/user/git/ansible-playbook
  executable location = /Users/user/git/ansible-playbook/venv/bin/ansible
  python version = 3.13.0 (main, Oct  9 2024, 16:56:22) [Clang 16.0.0 (clang-1600.0.26.3)] (/Users/user/git/ansible-playbook/venv/bin/python)
  jinja version = 3.1.4
  libyaml = False

COLLECTION VERSION

# /Users/kfrank/git/ansible-atlas/ansible_collections
Collection              Version  
----------------------- ---------
community.vmware        *        

CONFIGURATION

ANSIBLE_PIPELINING(/Users/user/git/ansible-playbook/ansible.cfg) = True
BECOME_ALLOW_SAME_USER(/Users/user/git/ansible-playbook/ansible.cfg) = True
CACHE_PLUGIN(/Users/user/git/ansible-playbook/ansible.cfg) = ansible.builtin.jsonfile
CACHE_PLUGIN_CONNECTION(/Users/user/git/ansible-playbook/ansible.cfg) = ./fact_cache
COLLECTIONS_PATHS(/Users/user/git/ansible-playbook/ansible.cfg) = ['/Users/user/git/ansible-playbook']
CONFIG_FILE() = /Users/user/git/ansible-playbook/ansible.cfg
DEFAULT_BECOME(/Users/user/git/ansible-playbook/ansible.cfg) = True
DEFAULT_FORCE_HANDLERS(/Users/user/git/ansible-playbook/ansible.cfg) = True
DEFAULT_FORKS(/Users/user/git/ansible-playbook/ansible.cfg) = 50
DEFAULT_GATHERING(/Users/user/git/ansible-playbook/ansible.cfg) = smart
DEFAULT_LOG_PATH(/Users/user/git/ansible-playbook/ansible.cfg) = /Users/user/git/ansible-playbook/logs
DEFAULT_MANAGED_STR(/Users/user/git/ansible-playbook/ansible.cfg) = This file is managed by ansible - local changes will be lost
DEFAULT_REMOTE_USER(/Users/user/git/ansible-playbook/ansible.cfg) = root
DEFAULT_TRANSPORT(/Users/user/git/ansible-playbook/ansible.cfg) = ssh
DEFAULT_VAULT_PASSWORD_FILE(/Users/user/git/ansible-playbook/ansible.cfg) = /Users/user/git/ansible-playbook/.vpass
DEPRECATION_WARNINGS(/Users/user/git/ansible-playbook/ansible.cfg) = True
RETRY_FILES_ENABLED(/Users/user/git/ansible-playbook/ansible.cfg) = False
SHOW_CUSTOM_STATS(/Users/user/git/ansible-playbook/ansible.cfg) = True
SYSTEM_WARNINGS(/Users/user/git/ansible-playbook/ansible.cfg) = True
USE_PERSISTENT_CONNECTIONS(/Users/user/git/ansible-playbook/ansible.cfg) = True

OS / ENVIRONMENT

MacOS

STEPS TO REPRODUCE

It worked when I recreated the venv for the 2nd time. Therefore I don't know what exactly caused this fluke. In both cases I did:

1. Get a MacBook with M3 Pro and macOS 15.1
2. Create a venv using python -m venv venv using Python 3.13.0
3. Enter that venv and install all requirements from bellows requirements.txt (source venv/bin/activate; pip install -r requirements.txt)
4. Create a test playbook for community.vmware.vmware_host_facts (see below)
5. Setup a fresh ESXi server at 10.0.0.1
6. Run that playbook and observe output.

- name: test
  hosts: localhost
  gather_facts: false
  serial: 1
  run_once: true
  tasks:
    - name: Gather vmware host facts
      community.vmware.vmware_host_facts:
        esxi_hostname: "10.0.0.1"
        hostname: "10.0.0.1"
        username: "{{ esxi_username }}"
        password: "{{ esxi_password }}"
        validate_certs: false
      register: host_facts

requirements.txt

pip>=24.3.1
setuptools>=75.3.0
wheel>=0.44.0
Cython>=3.0.11

ansible-core==2.17.2
ansible-lint==24.7.0
PyYAML==6.0.1
Jinja2==3.1.4
pyOpenSSL==24.2.1
requests==2.32.3
selinux==0.3.0
pywinrm==0.5.0
pexpect==4.9.0
ipaddress==1.0.23
cryptography==43.0.3
virtualenv==20.27.1
pypsexec==0.3.0
pyVim==3.0.3
pyVmomi==8.0.3.0.1

## The following requirements were added by pip freeze:
ansible-compat==24.9.1
attrs==24.2.0
black==24.10.0
bracex==2.5.post1
certifi==2024.8.30
cffi==1.17.1
charset-normalizer==3.4.0
click==8.1.7
distlib==0.3.9
distro==1.9.0
filelock==3.16.1
idna==3.10
importlib_metadata==8.5.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
markdown-it-py==3.0.0
MarkupSafe==3.0.2
mdurl==0.1.2
mypy-extensions==1.0.0
packaging==24.1
pathspec==0.12.1
platformdirs==4.3.6
ptyprocess==0.7.0
pycparser==2.22
Pygments==2.18.0
pyspnego==0.11.1
referencing==0.35.1
requests_ntlm==1.3.0
resolvelib==1.0.1
rich==13.9.4
rpds-py==0.21.0
ruamel.yaml==0.18.6
smbprotocol==1.14.0
subprocess-tee==0.4.2
urllib3==2.2.3
wcmatch==10.0
xmltodict==0.14.2
yamllint==1.35.1
zipp==3.20.2

EXPECTED RESULTS

Correct module output

ACTUAL RESULTS

TASK [Gather vmware host facts] *************************************************************************************************************************************************************************************************
task path: /Users/user/git/ansible-playbook/play.yml:16
Including module_utils file ansible/__init__.py
Including module_utils file ansible/module_utils/__init__.py
Including module_utils file ansible/module_utils/_text.py
Including module_utils file ansible/module_utils/basic.py
Including module_utils file ansible/module_utils/common/_utils.py
Including module_utils file ansible/module_utils/common/__init__.py
Including module_utils file ansible/module_utils/common/arg_spec.py
Including module_utils file ansible/module_utils/common/file.py
Including module_utils file ansible/module_utils/common/locale.py
Including module_utils file ansible/module_utils/common/parameters.py
Including module_utils file ansible/module_utils/common/collections.py
Including module_utils file ansible/module_utils/common/process.py
Including module_utils file ansible/module_utils/common/sys_info.py
Including module_utils file ansible/module_utils/common/text/converters.py
Including module_utils file ansible/module_utils/common/text/__init__.py
Including module_utils file ansible/module_utils/common/text/formatters.py
Including module_utils file ansible/module_utils/common/validation.py
Including module_utils file ansible/module_utils/common/warnings.py
Including module_utils file ansible/module_utils/compat/selinux.py
Including module_utils file ansible/module_utils/compat/__init__.py
Including module_utils file ansible/module_utils/distro/__init__.py
Including module_utils file ansible/module_utils/distro/_distro.py
Including module_utils file ansible/module_utils/errors.py
Including module_utils file ansible/module_utils/parsing/convert_bool.py
Including module_utils file ansible/module_utils/parsing/__init__.py
Including module_utils file ansible/module_utils/pycompat24.py
Including module_utils file ansible/module_utils/six/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/module_utils/vmware.py
Including module_utils file ansible/module_utils/common/_collections_compat.py
Including module_utils file ansible/module_utils/compat/version.py
Including module_utils file ansible_collections/__init__.py
Including module_utils file ansible_collections/community/__init__.py
Including module_utils file ansible_collections/community/vmware/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/module_utils/__init__.py
Including module_utils file ansible_collections/community/vmware/plugins/module_utils/vmware_rest_client.py
Using module file /Users/user/git/ansible-playbook/ansible_collections/community/vmware/plugins/modules/vmware_host_facts.py
Pipelining is enabled.
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: user
<127.0.0.1> EXEC /bin/sh -c '/Users/user/git/ansible-playbook/venv/bin/python && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/b8/jy21nnc92696ym8df0l6q5880000gn/T/ansible_community.vmware.vmware_host_facts_payload_9trabf74/ansible_community.vmware.vmware_host_facts_payload.zip/ansible_collections/community/vmware/plugins/module_utils/vmware.py", line 39, in <module>
    from pyVim import connect
ModuleNotFoundError: No module named 'pyVim'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "esxi_hostname": "10.0.0.1",
            "hostname": "10.0.0.1",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "properties": null,
            "proxy_host": null,
            "proxy_port": null,
            "schema": "summary",
            "show_datacenter": false,
            "show_tag": false,
            "username": "root",
            "validate_certs": false
        }
    },
    "msg": "Failed to import the required Python library (PyVmomi) on Mac.fritz.box's Python /Users/user/git/ansible-playbook/venv/bin/python. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

[ihumster]

Pyvmomi library build and tested only on 3.12 version maximum, if you look at https://github.com/vmware/pyvmomi

Try to use this collection with 3.12.

[mariolenz]

I agree with @ihumster, the problem is probably that pyVmomi isn't available for Python 3.13.

BTW I see pyVim==3.0.3 in your requirements.txt. This is not what the collection needs. The pyVim package that the collection needs is part of pyVmomi. Which, see above, isn't available for Python 3.13.

And I think we should ignore @lattaai9. The account has been created 2 hours ago. This looks suspicious to me. At the very least, this is some kind of spam. But it looks like some AI bullshit, and I'm not sure what is worse...

[agowa]

And I think we should ignore @lattaai9.

@mariolenz I already reported @lattaai9 for spamming to GitHub. Hope they're going to delete that crap soonish.

I agree with @ihumster, the problem is probably that pyVmomi isn't available for Python 3.13.

Could be, but still doesn't explain why it sometimes works and sometimes doesn't. Let's see if others are having similar problems and may have additional information then.

BTW I see pyVim==3.0.3 in your requirements.txt. This is not what the collection needs. The pyVim package that the collection needs is part of pyVmomi. Which, see above, isn't available for Python 3.13.

Is it bundled within pyVmomi or pulled in as a dependency?
For this issue I tried to freeze my dependencies so that it is more easily reproducible as it would pull in the very same versions even if one of the dependencies got updated in the mean time.

(And on another note, I'm still searching for a good way to vendor an entire python environment with dependencies that isn't docker, for air gapped environments as well as being able to reproduce todays environment in e.g. one year from now or when stuff got deleted/DMCAed/supply-chain-attacked/...).

[mariolenz]

I agree with @ihumster, the problem is probably that pyVmomi isn't available for Python 3.13.

Could be, but still doesn't explain why it sometimes works and sometimes doesn't. Let's see if others are having similar problems and may have additional information then.

Yes, this is indeed weird.

BTW I see pyVim==3.0.3 in your requirements.txt. This is not what the collection needs. The pyVim package that the collection needs is part of pyVmomi. Which, see above, isn't available for Python 3.13.

Is it bundled within pyVmomi or pulled in as a dependency? For this issue I tried to freeze my dependencies so that it is more easily reproducible as it would pull in the very same versions even if one of the dependencies got updated in the mean time.

I think it's part of pyVmomi, not just a dependency that's pulled in and installed. At least, that's how it looke like to me:

$ pip list | grep pyvmomi
pyvmomi                            8.0.3.0.1
$ pip show pyvmomi -f | grep pyVim
  pyVim/__init__.py
  pyVim/__pycache__/__init__.cpython-311.pyc
  pyVim/__pycache__/connect.cpython-311.pyc
  pyVim/__pycache__/sso.cpython-311.pyc
  pyVim/__pycache__/task.cpython-311.pyc
  pyVim/connect.py
  pyVim/sso.py
  pyVim/task.py
$