Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installing agavepy from poetry-built wheel ignores pre-releases #38

Open
ethho opened this issue Sep 30, 2021 · 0 comments · May be fixed by #37
Open

Installing agavepy from poetry-built wheel ignores pre-releases #38

ethho opened this issue Sep 30, 2021 · 0 comments · May be fixed by #37
Assignees
@ethho
Labels
bug Something isn't working

Comments

@ethho
Copy link
Collaborator

ethho commented Sep 30, 2021 •
edited
Loading

  • In order to poetry publish, need to set PyPi version for agavepy:
# resolves to 1.0.0a11
agavepy = {version = "^1.*", allow-prereleases = true}
  • make pytest-docker fails, complaining about no module agavepy.constants
    • 0.9.5 has no such submodule, but 1.0.0a11 and master branch do
  • The image, installing from the poetry build wheel at dist/reactors-*/*.whl, still has 0.9.5:
❯ git short
2cf3c94❯ make build
rm -rf dist
poetry build
Building reactors (1.0.0)
  - Building sdist
  - Built reactors-1.0.0.tar.gz
  - Building wheel
  - Built reactors-1.0.0-py3-none-any.whl
❯ docker build --progress plain -t tacc/reactors:1.0.0 .
#1 [internal] load build definition from Dockerfile
#1 sha256:5f201415d357feec42b5e4ed54cb6c0283270f5e69f45cc0caaacc318e299bb8
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:b52efc6261b23b77e1be194413222884b7fa657783bdaece3b3ccb242200312a
#2 transferring context: 34B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/python:3.7-buster
#3 sha256:32750e6511d1920b765363e67a50b83a52f2c66e92695e172a59a7ac15af0c5a
#3 DONE 0.4s

#4 [1/7] FROM docker.io/library/python:3.7-buster@sha256:19a352eaf0aabe88f2dd47c7cb73fccc8d2915a27d4a4b2c77c52db51240578c
#4 sha256:f68200e46ffd16a732fcdcd0547052d1d26a33931520a4f360c4303a768ab46a
#4 DONE 0.0s

#7 [internal] load build context
#7 sha256:dede657adff73507ddc3a3461f6294eb6cf9d826b3e38fdfa4dcea1d120b55e7
#7 transferring context: 73.33kB done
#7 DONE 0.0s

#5 [2/7] RUN apt-get update &&     apt-get install -y locales locales-all &&     apt-get clean
#5 sha256:eb7f97b1720db52543ec4e84cfc00519426c5b74bdaad60f16aaeb039a419983
#5 CACHED

#6 [3/7] RUN mkdir -p /mnt/ephemeral-01 &&     chmod a+rwx /mnt/ephemeral-01 &&     chmod g+rwxs /mnt/ephemeral-01
#6 sha256:9f06f2a2366fa17eb14cd3b29cbf7c8d374067bf7a074ddb661d0837a5647da6
#6 CACHED

#8 [4/7] ADD dist /tmp/reactors-dist
#8 sha256:e015c229e60acb8bdb9a03fd92b403656bac6a0c088cedd6953dab85173fab93
#8 DONE 0.0s

#9 [5/7] RUN pip install -q /tmp/reactors-dist/*.whl
#9 sha256:02adb07fcbd3cb8c88ee75a32dfdd196f847916191fe1a1ff178720589ebbe40
#9 7.998 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#9 DONE 8.3s

#10 [6/7] RUN python3 -m pip install cookiecutter &&     cd "$(dirname /default_actor_context)" &&     python3 -m cookiecutter --no-input -fc main --directory tacc_reactors_base               https://github.com/TACC-Cloud/cc-tapis-v2-actors.git name=/default_actor_context alias=/default_actor_context &&     cp /default_actor_context/reactor.py / &&     cp /default_actor_context/config.yml / &&     cp -r /default_actor_context/*_schemas / &&     python3 -m pip install --ignore-installed -r /default_actor_context/requirements.txt
#10 sha256:e1633bcffac18faa2f155509e1739fbcee9eb51efca21ec66509da76171be125
#10 0.583 Collecting cookiecutter
#10 0.636   Downloading cookiecutter-1.7.3-py2.py3-none-any.whl (34 kB)
#10 0.653 Requirement already satisfied: Jinja2<4.0.0,>=2.7 in /usr/local/lib/python3.7/site-packages (from cookiecutter) (3.0.1)
#10 0.694 Collecting python-slugify>=4.0.0
#10 0.707   Downloading python_slugify-5.0.2-py2.py3-none-any.whl (6.7 kB)
#10 0.768 Collecting jinja2-time>=0.2.0
#10 0.781   Downloading jinja2_time-0.2.0-py2.py3-none-any.whl (6.4 kB)
#10 0.838 Collecting click>=7.0
#10 0.851   Downloading click-8.0.1-py3-none-any.whl (97 kB)
#10 0.903 Collecting binaryornot>=0.4.4
#10 0.916   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
#10 0.921 Requirement already satisfied: requests>=2.23.0 in /usr/local/lib/python3.7/site-packages (from cookiecutter) (2.26.0)
#10 0.954 Collecting poyo>=0.5.0
#10 0.967   Downloading poyo-0.5.0-py2.py3-none-any.whl (10 kB)
#10 0.972 Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/site-packages (from cookiecutter) (1.16.0)
#10 1.007 Collecting chardet>=3.0.2
#10 1.019   Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
#10 1.041 Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/site-packages (from click>=7.0->cookiecutter) (4.8.1)
#10 1.046 Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from Jinja2<4.0.0,>=2.7->cookiecutter) (2.0.1)
#10 1.098 Collecting arrow
#10 1.116   Downloading arrow-1.1.1-py3-none-any.whl (60 kB)
#10 1.164 Collecting text-unidecode>=1.3
#10 1.186   Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
#10 1.208 Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests>=2.23.0->cookiecutter) (2021.5.30)
#10 1.210 Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests>=2.23.0->cookiecutter) (3.2)
#10 1.211 Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/site-packages (from requests>=2.23.0->cookiecutter) (2.0.6)
#10 1.212 Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests>=2.23.0->cookiecutter) (1.26.7)
#10 1.234 Requirement already satisfied: python-dateutil>=2.7.0 in /usr/local/lib/python3.7/site-packages (from arrow->jinja2-time>=0.2.0->cookiecutter) (2.8.2)
#10 1.235 Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/site-packages (from arrow->jinja2-time>=0.2.0->cookiecutter) (3.10.0.2)
#10 1.257 Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/site-packages (from importlib-metadata->click>=7.0->cookiecutter) (3.6.0)
#10 1.392 Installing collected packages: text-unidecode, chardet, arrow, python-slugify, poyo, jinja2-time, click, binaryornot, cookiecutter
#10 1.723 Successfully installed arrow-1.1.1 binaryornot-0.4.4 chardet-4.0.0 click-8.0.1 cookiecutter-1.7.3 jinja2-time-0.2.0 poyo-0.5.0 python-slugify-5.0.2 text-unidecode-1.3
#10 1.724 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#10 3.032 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#10 DONE 3.1s

#11 [7/7] WORKDIR /mnt/ephemeral-01
#11 sha256:cae21e86fd2ac78bcd179d23aa1c384b8bfd093f67eae08d27cb0799b4d7f2f4
#11 DONE 0.0s

#12 exporting to image
#12 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#12 exporting layers
#12 exporting layers 0.4s done
#12 writing image sha256:2ce05aabdc9a4a61e2a90bb5cf92b3c547b8faacf60e738427fb3804f0938c60 done
#12 naming to docker.io/tacc/reactors:1.0.0 done
#12 DONE 0.4s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
❯ docker run tacc/reactors:1.0.0 python3 -m pip freeze | grep agavepy
agavepy==0.9.5
  • This might have something to do with poetry #4405.
    • In other words, agavepy depedency resolves to 0.9.5 when built into a wheel
  • This seems to be the case: the simplified test case reveals the issue pretty well
❯ make build
❯ python3 -m venv ./virpy
❯ source virpy/bin/activate
❯ pip install dist/reactors-1.0.0-py3-none-any.whl
❯ pip show agavepy
Name: agavepy
Version: 0.9.5
Summary: SDK for Tapis
Home-page: https://github.com/TACC/agavepy
Author: Texas Advanced Computing Center
Author-email: [email protected], [email protected]
License: BSD
Location: /Users/eho/tacc/projects/sd2e/python-reactors/virpy/lib/python3.8/site-packages
Requires: py, requests-toolbelt, python-dateutil, backports.ssl-match-hostname, future, Jinja2, cloudpickle, requests, pytest, websocket-client, configparser, six
Required-by: reactors
❯ poetry show agavepy
name         : agavepy
version      : 1.0.0a11
description  : SDK for TACC Tapis (formerly Agave)

dependencies
- arrow >=0.15.5
- attrdict >=2.0.0
- cloudpickle >=1.3.0
- curlify >=2.2.1
- future >=0.18.2
- Jinja2 >=2.11.1
- petname >=2.6
- py >=1.8.1
- python-dateutil >=2.8.1
- python-dotenv >=0.11.0
- requests >=2.23.0
- requests-toolbelt >=0.9.1
- six >=1.12.0
- websocket-client >=0.57.0
❯ git short
034d900
  • Takeaway is that publishing to PyPi is blocked until one of the following occurs:
  • Next step is to surface #38 in poetry issue board
@ethho ethho self-assigned this Sep 30, 2021
@ethho ethho linked a pull request Sep 30, 2021 that will close this issue
Draft
@ethho ethho added the bug Something isn't working label Sep 30, 2021
@ethho ethho linked a pull request Sep 30, 2021 that will close this issue
Prepare v1.0.0 release #37
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ethho ethho

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Prepare v1.0.0 release TACC-Cloud/python-reactors
1 participant
@ethho