Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use {{ name }} in filename too #460

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

jakirkham
Copy link
Member

Description

Since {{ name }} is already templated elsewhere, template it in the filename of the archive too for consistency.

Note: This behavior was observed in comment ( conda-forge/staged-recipes#22395 (comment) )

Since `{{ name }}` is already templated elsewhere, template it in the
filename of the archive too for consistency.
@jakirkham jakirkham requested a review from a team as a code owner April 10, 2023 20:36
@jakirkham
Copy link
Member Author

Should add the other option would be to skip templating name everywhere and just inline it

@@ -132,8 +132,13 @@ def get_url_filename(metadata: dict, default: Optional[str] = None) -> str:

for pkg_url in metadata["urls"]:
if pkg_url["packagetype"] == "sdist":
name = metadata["info"]["name"]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT metadata is gotten by requesting JSON for a package on PyPI like so...

import json
import requests


name = "dask-cuda"

req = requests.get(f"https://pypi.org/pypi/{name}/json")
metadata = json.loads(req.content)

assert metadata["info"]["name"] == name

This appears valid when looking at dask-cuda on PyPI

Am I missing something?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the fixture might be incomplete? (log)

metadata = {'info': {'version': '1.2.3'}, 'urls': [{'filename': 'foo_file-1.2.3.zip', 'packagetype': 'sdist'}]}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have often wished the upstream Warehouse data model was captured somewhere other than in the code for warehouse (e.g. an OpenAPI spec, some TypedDicts, really anything...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good catch! Thanks Nick 🙏

Wasn't thinking about fixtures for some reason 🤦‍♂️

Yeah was really hoping I could find some docs describing this somewhere, but was unable 😞

Copy link

@bollwyvl bollwyvl Apr 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I remember, pypi converts the _ to -, but it does not work for all
packages that were initially specified as having the _ will just work with it. pypi api is a bit confusing

@jakirkham
Copy link
Member Author

Also looks like semver version 3.0.0 changed from isvalid to is_valid in PR ( python-semver/python-semver#368 ), which is causing some unrelated test failures

@bollwyvl
Copy link

I've found that an increasing number of packages' build chains will start spitting out *_*.tar.gz without much warning, so using the "PyPI normalized" name is less predictable than it once was. I guess i'd rather see the default be to remove {{ name }} entirely.

@jakirkham
Copy link
Member Author

Am ok with that. Mainly want to be consistent with whichever approach we choose

@marcelotrevisani
Copy link
Member

Hi folks, sorry for my delay.
I was thinking in the past to use the pypi url that gives the filename and all and just replace the version by the jinja expression {{ version}}

@marcelotrevisani
Copy link
Member

I am a bit busy these days, I don't have much free time to take a proper look at this
I believe in a few days I can do a proper investigation into this. Thanks!

@jakirkham jakirkham closed this Jul 31, 2023
@jakirkham jakirkham reopened this Jul 31, 2023
@jakirkham
Copy link
Member Author

Toggling for CI

@marcelotrevisani
Copy link
Member

is it still relevant? I remember that it is not using {{ name }} in the filename to avoid problems with - or _

@jakirkham
Copy link
Member Author

Yes as we are still handling name inconsistently

That said, if we would rather inject the name and not use Jinja in the URL, that is also an option. Please see PR: #548

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants