Support for PEP 735 dependency groups

[cthoyt]

What's the problem this feature will solve?

PEP 735 introduced dependency groups, which are complementary to optional dependencies in that dependency groups might not correspond to features in the package, but rather be something like development or release dependencies.

You can install something with dependency groups like this:

$ pip install --dependency-groups=tests,typing

ReadTheDocs currently supports specifying optional dependencies (c.f., https://docs.readthedocs.io/en/stable/config-file/v2.html#packages) with configuration like the following, where docs probably has sphinx, sphinx-rtd-theme, and other sphinx plugins.

version: 2

python:
  install:
    - method: pip
      path: .
      extra_requirements:
        - docs

Describe the solution you'd like

I'd like an additional configuration in the install dictionary that works similar to extra_requirements that would correspond to dependency groups. Let's say I have used the PEP 735 definition of my docs, like in this abbreviated pyproject.toml:

[dependency-groups]
tests = [
    "pytest",
    "coverage",
]
docs = [
    "sphinx>=8",
    "sphinx-rtd-theme>=3.0",
    "sphinx_automodapi",
]

then I would want to define it with an alternate key, like dependency_groups (just a suggestion for the name):

version: 2

python:
  install:
    - method: pip
      path: .
      dependency_groups:
        - docs

Alternative solutions

This approach isn't required to get the intended results, which is of course to install the right extra dependencies to get my docs to build. However, the main goal is to better organize my metadata, and not to expose sphinx as an "extra" on PyPI, which doesn't exactly fit the spirit of extras/optional dependencies

Another in-progress solution (mentioned in #11766 (comment)) is #11710, which will allow for:

version: 2

build:
  jobs:
    install:
      - pip install --dependency-groups=test,typing 

Additional context

This is motivated by recent improvements in pip, uv, and tox! I have an (almost) working demo in my cookiecutter project which shows how to update configuration properly to support this cthoyt/cookiecutter-snekpack#32

I am not familiar at all with the RTD stack, but if there's a way I can contribute to coding this up, please let me know :)

Nitty-gritty

Here are a few places which probably would be part of a theoretical implementation:

readthedocs.org/readthedocs/config/models.py

Lines 77 to 81 in 404d82a

	class PythonInstall(Base):
	__slots__ = (
	"path",
	"method",
	"extra_requirements",

I'd simply add a new slot dependency_groups here

readthedocs.org/readthedocs/doc_builder/python_environments.py

Lines 66 to 67 in 404d82a

	if install.extra_requirements:
	extra_req_param = "[{}]".format(",".join(install.extra_requirements))

here's how I'd update this:

# Added these next lines
dependency_group_args = []
if install.dependency_groups:
    # not clear if the equals is necessary or if
    # this can be broken into two parts
    dependency_group_args.append("--dependency-groups={}".format(",".join(install.dependency_groups))

extra_req_param = ""
if install.extra_requirements:
    extra_req_param = "[{}]".format(",".join(install.extra_requirements))
self.build_env.run(
    self.venv_bin(filename="python"),
    "-m",
    "pip",
    "install",
    "--upgrade",
    "--upgrade-strategy",
    "only-if-needed",
    "--no-cache-dir",
    "{path}{extra_requirements}".format(
        path=local_path,
        extra_requirements=extra_req_param,
    ),
    *dependency_group_args,
    cwd=self.checkout_path,
    bin_path=self.venv_bin(),
)

As a minor note, I would also do a bit of refactoring to store all of the args into the list and then splat all of them into run()

[humitos]

Hi @cthoyt, thanks for opening this issue.

I don't think we will support this soon since it requires a lot of extra work from our side and we are focusing ourselves on different features currently.

However, this use case will be a really good fit for the work we are doing on #11710. Once that PR gets merged, you will be able to write something like:

version: 2

build:
  jobs:
    install:
      - pip install --dependency-groups=test,typing 

[cthoyt]

@humitos super, I hope this brings a lot of simplification to your work!

I will be happy to test run that this is working properly when #11710 is ready.

Feel free to close or leave this issue open in case anyone else comes looking for it.

[fepegar]

Happy to test as well. I'm having the same problem (build) at this PR:

• Split extra dependencies and dependency groups fepegar/torchio#1238

[agjohnson]

You should be able to test this out, #11710 was released this week and it seems to behaving well so far. We'd be curious to know how this goes and if this is an easy way to support custom/alternative installation methods.

[fepegar]

It seems that pip doesn't have --dependency-groups (yet). I would typically use uv for this. Do you have any tips?

[fepegar]

I guess a more general question is: how does one use the new feature?

[agjohnson]

Yeah we don't have any serious docs on this yet, you'll have to glue some pieces together here.

I haven't followed dependency groups in pip, but if it's a case of pip being out of date, you might also need to upgrade pip in your own project for now. The version we install might be lagging a little bit.

For uv, support isn't built in but can be achieved with some extra setup:
#11289

In the end, you should have something like:

build:
  jobs:
    install:
      - pip install --dependency-groups=tests,typing

That is just a rough example though.

[fepegar]

Thank you, @agjohnson. This seems to have worked:

build:
  os: ubuntu-22.04
  tools:
    python: "3.12"
  jobs:
    install:
      - pip install .
      - pip install dependency-groups
      - pip-install-dependency-groups doc