Expose a way to exclude deps - option 2

[sivel]

Based on #647

---
version: 3.1

images:
  base_image:
    name: quay.io/centos/centos:stream9

dependencies:
  python_interpreter:
    package_system: python3.11
    python_path: /usr/bin/python3.11

  ansible_core:
    package_pip: ansible-core

  ansible_runner:
    package_pip: ansible-runner

  galaxy:
    collections:
      - name: community.docker
      - name: ansible.netcommon

  exclude:
    python:
      - docker
    system:
      - python3-Cython 

TODO:

• python excludes should work
• excludes should not apply to user deps
• system excludes should work
• should there be a means to exclude collections?

NOTES:

• Only supports exclusion of first level deps, does not offer a way to exclude deps of deps

[sivel]

Some additional care is probably required here. This will have issues currently with things like:

• URLs (git+..., https://...)
• -r extra-requirements.txt or any other random pip flag

In this case, we probably just need to skip the exclude checks. But we can't have it both ways, in that we can't ignore requirements parsing, and still do requirements parsing, so URLs and such just wouldn't be supported. 🙂

The regex is also using the pep503 info, but I can't think of reasons it wouldn't work with RPM package names.

[sivel]

This may not actually be a problem with -r but would still apply to URLs

[sivel]

We need to kill off fancy lines, and just spit back out a concatenation, but not consolidate duplicates.

[sivel]

Suggested change

	REQ_NAME_RE = re.compile(r'^([-\w.]+)')
	REQ_NAME_RE = re.compile(r'^([^-][-\w.]+)')

[nitzmahone]

This is the best approach that seems the least likely to break existing usages, while still allowing for various "escape hatch" impls... Probably a few more bits to implement to fully close the loop:

• A from_collection key for exclude, to allow a list of one or more collection names from which to completely ignore all deps. It's a big hammer, but would make things like "I only want to get my deps from downstream-packaged sources" at least possible without requiring EE creators to manually merge and exclude all the deps using the python/system/galaxy keys.
• A docs note that Python requirements in collections should be limited to features defined by PEP508, and that any deviation from that will be considered undefined/unsupported behavior, even if it happens to work today.
• A suggestion to partner eng that PEP508-only requirements validation might be a good candidate for a certification gate (they could easily base such a thing off Drop requirements-parser in favor of pep508 #645).

[nitzmahone]

Since we're abusing a combination of other tools in the absence of a first-class YAML "schema" for EE definitions , it's probably better to just support this in the existing version 3 schema, along with a small docs note that the exclude feature requires Builder >= 3.1 (and will be an error on < 3.1).

[Shrews]

Closing this one and continuing work in #664 so as not to annoy @sivel with noise on this one.