Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with Poetry (#1) #16

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Build with Poetry (#1) #16

wants to merge 13 commits into from

Conversation

drernie
Copy link

@drernie drernie commented Aug 13, 2024

  • Fix deprecated read_schema in new pyarrow versions

  • Use poetry

  • rename project folder

  • Python CI

  • lint

  • make test

* Fix deprecated read_schema in new pyarrow versions

https://github.com/dacort/athena-federation-python-sdk/pull/13/files

* Create .flake8

* Use poetry

Update versions

* rename project folder

* Python CI

* lint

* remove subfolders

* bump CI versions

* simplify package name

* remove flake8

AttributeError: 'EntryPoints' object has no attribute 'get'
> Because importlib-metadata releases v5.0.0 yesterday which it remove deprecated endpoint
https://stackoverflow.com/questions/73929564/entrypoints-object-has-no-attribute-get-digital-ocean

* megalinter fixes

* cleanup lock

* drop AWS Role from CI

* make test

* test coverage

* lint Makefile

* .PHONY all

* Update Makefile

* disable MAKEFILE_CHECKMAKE
@drernie
Copy link
Author

drernie commented Aug 13, 2024

Hope this isn't too much divergence.

dependabot bot and others added 3 commits August 12, 2024 18:32
Bumps [black](https://github.com/psf/black) from 20.8b1 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/commits/24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* pytest-watcher

* ignore coverage files

* test handler

* pass test_handler
* TABLE_DEF

* test_ReadRecordsRequest

* expect coverage
@dacort
Copy link
Owner

dacort commented Aug 13, 2024

Rock on! There's a lot here, but let me try to take a look in the next day or two!

* update README

* arm64v8/python:3.12.5

* docker-poetry-config

* switch ports

* run minimal dockerfile

* smarter

* remove cache

* --no-root

* move example into module

* expose lambda_handler on port 9000

* pull out example

* IMG = local/athena-federation

* image build --platform=linux/amd64

* container run $(PLAT)

* ARCH = amd64

* example.handler.sample_handler

* install from wheel

* make docker works!

docker-stop docker-build docker-detached lambda-ping lambda-list-schemas

* cleanup Dockerfile

* cov-report

* WORKDIR

* main coverage
pyproject.toml Outdated
name = "athena-federation"
version = "0.1.2"
description = "Unofficial Python SDK for Athena Federation"
authors = ["Damon P. Cortesi <[email protected]>", "Ernest Prabhakar <[email protected]>"]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
authors = ["Damon P. Cortesi <DamonPCortesi@gmail.com>", "Ernest Prabhakar <[email protected]>"]
authors = ["Damon P. Cortesi <d.lifehacker@gmail.com>", "Ernest Prabhakar <[email protected]>"]

pyproject.toml Outdated
Comment on lines 8 to 10
homepage = "https://github.com/quiltdata/athena-federation-python-sdk"
repository = "https://github.com/quiltdata/athena-federation-python-sdk"
documentation = "https://github.com/quiltdata/athena-federation-python-sdk"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change these back to dacort repo?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
homepage = "https://github.com/quiltdata/athena-federation-python-sdk"
repository = "https://github.com/quiltdata/athena-federation-python-sdk"
documentation = "https://github.com/quiltdata/athena-federation-python-sdk"
homepage = "https://github.com/dacort/athena-federation-python-sdk"
repository = "https://github.com/dacort/athena-federation-python-sdk"
documentation = "https://github.com/dacort/athena-federation-python-sdk"

Dockerfile Outdated

COPY example/ ./
RUN ls ./
COPY example/ ./example
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason when I ran the make lambda-ping below, I got back this error:

Unable to import module 'example.handler': No module named 'example'

Looking into why, but feel free to comment/fix if you know.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a missing dependency in the Makefile for that action. I'll take a look.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh! Now I'm seeing this error everywhere. I have this horrible feeling I never got it to work, but simply mis-read the error message...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whew. LAMBDA_TASK_ROOT fixes it. Rolling into 0.1.3

Copy link
Owner

@dacort dacort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 Thank you so much for updating the project dependencies/build process. Just a few minor things.

Dockerfile Outdated

COPY example/ ./
RUN ls ./
COPY example/ ./example
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
COPY example/ ./example
COPY example/ ${LAMBDA_TASK_ROOT}/example

Per https://docs.aws.amazon.com/lambda/latest/dg/python-image.html LAMBDA_TASK_ROOT is the default path for modules.

* fix ownership

* use toml for version

* LAMBDA_TASK_ROOT

* bump version

* revert use of toml

* hacked Dockerfile that works

* minimal Dockerfil

* even more minimal
* support types

* lint types

* fix type errors

* retrofit CHANGELOG

* extract transpose_data

* ignore from_arrays type error

* fix coverage

* cleanup

* final lint
* Move py.typed into package folder

* bump version

* reenable PYTHON_MYPY

* lint all files

To match MYPY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants