Sort by repo of OCA

[AnizR]

It would be great to have the name of repository such as OCA/_repo_name when sorting addons.

An idea would be to exploit the 'website' in the manifest.

[sbidoul]

Unfortunately, I realize we don't have the manifests of the dependencies available when running in pre-commit.

However the website field of the Odoo manifest is stored into the Home-Page metadata of the distribution package, so we can obtain that on PyPI. The script below illlustrates how to do this.

One attention point is that it will download a bit more data from the index than the simple HEAD request we currently make. When run locally, this will be fine due to the cache, but this may mean we'll need to use the GitLab/GitHub CI cache when pre-commit runs in CI.

# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "manifestoo_core",
#   "mousebender",
#   "packaging",
#   "httpx",
# ]
# ///

from email.parser import Parser as MetadataParser

import httpx
from manifestoo_core.odoo_series import OdooSeries
from manifestoo_core.metadata import addon_name_to_distribution_name
from mousebender import simple
from packaging.specifiers import SpecifierSet
from packaging.utils import parse_wheel_filename

INDEX_URL = "https://pypi.org/simple/"


def get_oca_repo(addon_name: str, odoo_series: OdooSeries) -> str | None:
    specifier = SpecifierSet(f"=={odoo_series.value}.*")
    distribution_name = addon_name_to_distribution_name(addon_name, odoo_series)
    # get avaialble releases
    project_url = simple.create_project_url(INDEX_URL, distribution_name)
    response = httpx.get(project_url, headers={"Accept": simple.ACCEPT_JSON_V1})
    if response.status_code == 404:
        # project not found
        return None
    response.raise_for_status()
    data = response.text
    content_type = response.headers["Content-Type"]
    project_details = simple.parse_project_details(data, content_type, distribution_name)
    # find the first version that matches the requested Odoo version;
    # we assume all releases come from the same repo for a given Odoo series
    for file in project_details["files"]:
        if file.get("yanked"):
            continue
        filename = file["filename"]
        if not filename.endswith(".whl"):
            continue
        _, version, _, _ = parse_wheel_filename(filename)
        if specifier.contains(version, prereleases=True):
            # found a release that matches the requested Odoo version
            break
    else:
        # no release found that matches the requested Odoo version
        return None
    if not file.get("data-dist-info-metadata"):
        return "UNKNOWN"
    metadata_url = file["url"] + ".metadata"
    response = httpx.get(metadata_url)
    response.raise_for_status()
    metadata = MetadataParser().parsestr(response.text)
    return metadata.get("Home-Page")


print(get_oca_repo("mis_builder", OdooSeries.v16_0))

[AnizR]

Thanks for your suggestion!

I opted to use the JSON API of PyPI, as it appeared to be a simpler approach compared to parsing the wheel metadata. Do you have any thoughts on that?

One attention point is that it will download a bit more data from the index than the simple HEAD request we currently make. When run locally, this will be fine due to the cache, but this may mean we'll need to use the GitLab/GitHub CI cache when pre-commit runs in CI.

I already had a cache setup to retrieve the 'category' of an add-on, so I simply reused that. I'm unsure if I need to do anything further for the CI cache—should this be defined within the repository or when setting up CI?

[sbidoul]

I opted to use the JSON API of PyPI, as it appeared to be a simpler approach compared to parsing the wheel metadata. Do you have any thoughts on that?

The drawback of the PyPI JSON API is that it is not standardized, so other package indexes than PyPI may not support it.

CI cache—should this be defined within the repository or when setting up CI?

That's when setting up the CI indeed.

[AnizR]

The drawback of the PyPI JSON API is that it is not standardized, so other package indexes than PyPI may not support it.

Oh, okay, you made your point 😄 I'll get back to the parsing of the Metadata as suggeted.

[sbidoul]

To parse the metadata, it would be better to use packaging.metadata rather than email.parser: https://packaging.pypa.io/en/stable/metadata.html