chore(grpc-asyncio): replace hatch with poetry

Since poetry is used in our other python libraries, and hatch is
hard to understand (and to get working), we replace hatch with
poetry.

As a nifty side-effect, the Docker container is now executed as
a user with the same uid/gid as the host user, resulting in
files created with the correct permissions in the mounted host
volume.

.github/workflows/code-qa.yaml

@@ -3,40 +3,32 @@ name: Code-QA
 on: push
 
 jobs:
-  ruff-lint:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: chartboost/ruff-action@v1
 
-  ruff-format:
-    runs-on: ubuntu-latest
+jobs:
+  python-ci:
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
+        poetry-version: ["1.8"]
+        os: [ubuntu-22.04, ubuntu-24.04, macos-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
-      - uses: chartboost/ruff-action@v1
+      - uses: actions/setup-python@v4
         with:
-          src: "."
-          args: format --check --diff
-
-  pylint:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v4
-      - name: Setup Python
-        uses: actions/setup-python@v5
+          python-version: ${{ matrix.python-version }}
+      - uses: abatilo/actions-poetry@v2
         with:
-          python-version: 3.8
-          architecture: x64
-      - name: Install pylint
-        run: |
-          pip install pylint
-          pip install python/remotivelabs-broker
-          pip install python/remotivelabs-broker[default]
-      - name: Run pylint
-        run: |
-          cd python/remotivelabs-broker
-          pylint **/*.py
+          poetry-version: ${{ matrix.poetry-version }}
+      - run: poetry self add 'poethepoet[poetry_plugin]'
+      - run: poetry install
+      - name: run pytest
+        run: poetry poe test-local
+      - name: run ruff
+        run: poetry poe format-check
+      - name: run mypy
+        run: poetry poe mypy-check
 
   rust-cargo:
     name: Rust cargo tasks

python/README.md

@@ -1,60 +1,72 @@
-# RemotiveLabs Python libraries
+# RemotiveLabs Broker API Python SDK
 
-## Usage
+`remotivelabs-broker` - Python SDK for interacting with the RemotiveLabs Broker.
 
-Python RemotiveBroker API
+Published to PyPI on [https://pypi.org/project/remotivelabs-broker/](https://pypi.org/project/remotivelabs-broker/)
 
-https://pypi.org/project/remotivelabs-broker/
+## Getting started
 
-Published on [PyPi](https://pypi.org/) with the prefix `remotivelabs-` and with the user [remotivelabs](https://pypi.org/user/remotivelabs/).
-All packages use [Hatch](https://hatch.pypa.io/) for packaging and publishing.
+Prerequisites:
 
+```bash
+# Install poetry (and optionally poe plugin)
+pipx install poetry
+poetry self add 'poethepoet[poetry_plugin]'
+```
 
+Build and run:
 
-## Development
+```bash
+cd python/remotivelabs-broker
 
-Install hatch on your development computer:
+# Install dependencies in a virtualenv
+poetry install
 
-    pip install hatch
+# Build the library (output in dist/)
+poetry build
 
-Any project related information is stored in `pyproject.toml`.
+# test local (local test does not require a running broker)
+poetry poe test-local
 
-Hatch supplies a basic set of tools for development.
-All operations should be done in an virtual environment created by hatch.
+# test server (server test requires a running broker)
+poetry poe test-server
 
-Go to the directory of the module which you are going to be working on.
-For example:
+# test all
+poetry poe test
+```
 
-    cd python/remotivelabs-broker
+If you need to (re)generate protobuf stubs, see [Building](#building).
 
-Create a local environment.
+## Building
 
-    hatch env create
+Building the complete package, including protobuf stubs and documentation, is done using a docker container:
 
-Start a shell in the environment.
+```bash
+cd python/remotivelabs-broker
+./docker-build.sh
+```
 
-    hatch shell
+## Versioning
 
-While in this virtual environment the library is available as a python module without needing to be installed.
+Update the package version by editing the following files:
+- `pyproject.toml`
+- `remotivelabs/broker/__about__.py`
 
-Build the library:
+Follow [Semantic versioning](https://semver.org/). Beta versions should be suffixed with `b*`, example `0.2.0b1`.
 
-    hatch build
+## Publishing
 
-Update the package version by editing the file `__about__.py`.
-This version stamp will be used in Pypi.
-Follow [Semantic versioning](https://semver.org/).
+Published to PyPI on [https://pypi.org/project/remotivelabs-broker/](https://pypi.org/project/remotivelabs-broker/):
 
-Testing, with coverage:
+All RemotiveLabs libraries:
+- share the same `remotivelabs` [namespace package](https://peps.python.org/pep-0420/).
+- use the `remotivelabs-` prefix in the library name.
+- are published with the [remotivelabs](https://pypi.org/user/remotivelabs/) user.
 
-    hatch run cov
-
-Without coverage:
-
-    hatch run no-cov
-
-While in the hatch virtual environment.
-Run the Python debugger:
-
-    python -m pdb ...
+```bash
+# Get token from Johan Rask
+poetry config pypi-token.pypi <my-token>
 
+# find username and password in less secret location
+poetry publish
+```

python/remotivelabs-broker/README.md

@@ -12,7 +12,6 @@
 
 - [Installation](#installation)
 - [Examples](#examples)
-- [Build and Publish](#build-and-publish)
 - [License](#license)
 
 ## Installation
@@ -25,52 +24,6 @@ Install into your Python environment with _PIP_:
 
 Examples using this library are found in the [Remotive Labs samples repository](https://github.com/remotivelabs/remotivelabs-samples).
 
-
-## Develop and test locally
-
-Put version in `__about__.py`. Beta versions should be suffixed with `b*`, example `0.2.0b1`
-
-```bash
-# For initial build and generation of proto stubs, documentation and distribution package
-./docker-build.sh
-
-Once stubs are generated you can use `hatch` to (re)-build distribution package. Use e.g. [pipx](https://github.com/pypa/pipx) to install `hatch`.
-
-```bash
-# install hatch in isolated env
-pipx install hatch
-
-# rebuild
-hatch build
-> [wheel]
-> dist/remotivelabs_broker-<version>-py3-none-any.whl
-
-# you can now install this library version in a virtualenv by opening a new terminal and doing:
-python3 -m venv .venv
-source .venv/bin/activate
-python3 -m pip install --upgrade pip
-pip3 install dist/remotivelabs_broker-<version>-py3-none-any.whl
-
-# test that the package is importable
-python -c "import remotivelabs.broker.sync as br; print(dir(br.SignalCreator))"
-
-# when you are done, exit the virtualenv
-deactivate
-```
-
-## Build and publish
-
-Make sure to put version in `__about__.py`. Beta versions should be suffixed with `b*`, example `0.2.0b1`
-
-```
-./docker-build.sh
-hatch publish
-```
-find username and password in less secret location
-or use
-```
-hatch publish --user __token__
-```
 ## License
 
 `remotivelabs-broker` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.

python/remotivelabs-broker/docker-build.sh

@@ -13,9 +13,4 @@ docker build -t "${TAG}" -f "${SCRIPT_DIR}/docker/Dockerfile" "${SCRIPT_DIR}"
 ARGS="-it"
 [ "${NO_TTY}" == "true" ] && ARGS="-i"
 
-docker run \
-    -u "$(id -u):$(id -g)" \
-    -v "${SCRIPT_DIR}/../../:/app"  \
-    -e "protofile=*.proto"  \
-    -w /app  \
-    "${ARGS}" "${TAG}"
+docker run -v "${SCRIPT_DIR}/../../:/app" "${ARGS}" "${TAG}"

python/remotivelabs-broker/docker/Dockerfile

@@ -1,22 +1,27 @@
-FROM ubuntu:22.04
+FROM python:3.10-slim
 
-ENV protofile=*.proto
+ENV protofile=*.proto \
+  # python config
+  PYTHONFAULTHANDLER=1 \
+  PYTHONUNBUFFERED=1 \
+  PYTHONHASHSEED=random \
+  # pip config
+  PIP_NO_CACHE_DIR=off \
+  PIP_DISABLE_PIP_VERSION_CHECK=on \
+  PIP_DEFAULT_TIMEOUT=100 \
+  # poetry config
+  POETRY_VERSION=1.8
 
 RUN apt-get update && \
     apt-get install -y --no-install-recommends \
-        python3.11 \
-        python3-pip \
         wget \
         unzip && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
-RUN pip install --no-cache-dir \
-    grpcio-tools==1.44.0 \
-    mypy-protobuf==3.3.0 \
-    hatch==1.12.0 \
-    pdoc==14.6.1 \
-    json-schema-for-humans==1.0.2
+# See https://python-poetry.org/docs/#ci-recommendations
+RUN pip install "poetry==$POETRY_VERSION"
+RUN poetry self add 'poethepoet[poetry_plugin]'
 
 # install the protobuf compiler
 COPY --chmod=0755 docker/download_protoc.sh /tmp/download_protoc.sh
@@ -26,9 +31,11 @@ RUN /tmp/download_protoc.sh
 COPY --chmod=0755 docker/download_protoc_gen_doc.sh /tmp/download_protoc_gen_doc.sh
 RUN /tmp/download_protoc_gen_doc.sh
 
+# Set up build script and entrypoint. It is possible to customize arguments to it using CMD or from command line
+COPY --chmod=755 docker/entrypoint.sh /usr/local/bin/entrypoint.sh
+COPY --chmod=755 docker/build.sh /usr/local/bin/build.sh
+
 VOLUME /app
-VOLUME /output
+WORKDIR /app
 
-# Set up build script as entrypoint. It is possible to customize arguments to it using CMD or from command line
-COPY --chmod=755 docker/entrypoint.sh /usr/local/bin/entrypoint.sh
 ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]