Skip to content

Commit

Permalink
chore(grpc-asyncio): replace hatch with poetry
Browse files Browse the repository at this point in the history
Since poetry is used in our other python libraries, and hatch is
hard to understand (and to get working), we replace hatch with
poetry.

As a nifty side-effect, the Docker container is now executed as
a user with the same uid/gid as the host user, resulting in
files created with the correct permissions in the mounted host
volume.
  • Loading branch information
joekickass committed Sep 12, 2024
1 parent 9783b46 commit a8c9b32
Show file tree
Hide file tree
Showing 19 changed files with 1,436 additions and 356 deletions.
30 changes: 0 additions & 30 deletions .github/workflows/build-docs-mypy.yaml

This file was deleted.

53 changes: 53 additions & 0 deletions .github/workflows/build-remotivelabs-broker-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Build remotivelabs-broker package and documentation

on: push

jobs:
build:
name: Build remotivelabs-broker

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Build package and docs
env:
NO_TTY: true
run: ./python/remotivelabs-broker/docker-build.sh
- id: build_output
run: echo "whl_path=$(ls python/remotivelabs-broker/dist/*.whl)" >> $GITHUB_OUTPUT

- uses: actions/upload-artifact@v4
with:
name: remotivelabs-broker-whl
path: ${{ steps.build_output.outputs.whl_path }}
if-no-files-found: error
retention-days: 1
overwrite: true

test:
name: Install remotivelabs-broker
needs: build

strategy:
matrix:
python-version: ["3.8", "3.9", "3.10"]
os: [ubuntu-22.04, ubuntu-24.04, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}

steps:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- uses: actions/download-artifact@v4
with:
name: remotivelabs-broker-whl

- name: Install remotivelabs-broker
run: pip install *.whl
53 changes: 53 additions & 0 deletions .github/workflows/code-qa-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Code QA CI

on: push

jobs:
python-ci:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10"]
defaults:
run:
working-directory: ./python/remotivelabs-broker

steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- uses: abatilo/actions-poetry@v2
- run: poetry self add 'poethepoet[poetry_plugin]'

- name: Generate stubs (and build package)
env:
NO_TTY: true
run: ./docker-build.sh

- run: poetry install
- name: run pytest
run: poetry poe test-local
- name: run ruff
run: poetry poe format-check
- name: run mypy
run: poetry poe mypy-check

rust-cargo:
name: Rust cargo tasks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Build and run tests
run: |
pushd rust/remotivelabs-broker
cargo test
popd
- name: Check format
run: |
pushd rust/remotivelabs-broker
cargo fmt --check
popd
56 changes: 0 additions & 56 deletions .github/workflows/code-qa.yaml

This file was deleted.

84 changes: 48 additions & 36 deletions python/README.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,72 @@
# RemotiveLabs Python libraries
# RemotiveLabs Broker API Python SDK

## Usage
`remotivelabs-broker` - Python SDK for interacting with the RemotiveLabs Broker.

Python RemotiveBroker API
Published to PyPI on [https://pypi.org/project/remotivelabs-broker/](https://pypi.org/project/remotivelabs-broker/)

https://pypi.org/project/remotivelabs-broker/
## Getting started

Published on [PyPi](https://pypi.org/) with the prefix `remotivelabs-` and with the user [remotivelabs](https://pypi.org/user/remotivelabs/).
All packages use [Hatch](https://hatch.pypa.io/) for packaging and publishing.
Prerequisites:

```bash
# Install poetry (and optionally poe plugin)
pipx install poetry
poetry self add 'poethepoet[poetry_plugin]'
```

Build and run:

## Development
```bash
cd python/remotivelabs-broker

Install hatch on your development computer:
# Install dependencies in a virtualenv
poetry install

pip install hatch
# Build the library (output in dist/)
poetry build

Any project related information is stored in `pyproject.toml`.
# test local (local test does not require a running broker)
poetry poe test-local

Hatch supplies a basic set of tools for development.
All operations should be done in an virtual environment created by hatch.
# test server (server test requires a running broker)
poetry poe test-server

Go to the directory of the module which you are going to be working on.
For example:
# test all
poetry poe test
```

cd python/remotivelabs-broker
If you need to (re)generate protobuf stubs, see [Building](#building).

Create a local environment.
## Building

hatch env create
Building the complete package, including protobuf stubs and documentation, is done using a docker container:

Start a shell in the environment.
```bash
cd python/remotivelabs-broker
./docker-build.sh
```

hatch shell
## Versioning

While in this virtual environment the library is available as a python module without needing to be installed.
Update the package version by editing the following files:
- `pyproject.toml`
- `remotivelabs/broker/__about__.py`

Build the library:
Follow [Semantic versioning](https://semver.org/). Beta versions should be suffixed with `b*`, example `0.2.0b1`.

hatch build
## Publishing

Update the package version by editing the file `__about__.py`.
This version stamp will be used in Pypi.
Follow [Semantic versioning](https://semver.org/).
Published to PyPI on [https://pypi.org/project/remotivelabs-broker/](https://pypi.org/project/remotivelabs-broker/):

Testing, with coverage:
All RemotiveLabs libraries:
- share the same `remotivelabs` [namespace package](https://peps.python.org/pep-0420/).
- use the `remotivelabs-` prefix in the library name.
- are published with the [remotivelabs](https://pypi.org/user/remotivelabs/) user.

hatch run cov

Without coverage:

hatch run no-cov

While in the hatch virtual environment.
Run the Python debugger:

python -m pdb ...
```bash
# Get token from Johan Rask
poetry config pypi-token.pypi <my-token>

# find username and password in less secret location
poetry publish
```
47 changes: 0 additions & 47 deletions python/remotivelabs-broker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@

- [Installation](#installation)
- [Examples](#examples)
- [Build and Publish](#build-and-publish)
- [License](#license)

## Installation
Expand All @@ -25,52 +24,6 @@ Install into your Python environment with _PIP_:

Examples using this library are found in the [Remotive Labs samples repository](https://github.com/remotivelabs/remotivelabs-samples).


## Develop and test locally

Put version in `__about__.py`. Beta versions should be suffixed with `b*`, example `0.2.0b1`

```bash
# For initial build and generation of proto stubs, documentation and distribution package
./docker-build.sh

Once stubs are generated you can use `hatch` to (re)-build distribution package. Use e.g. [pipx](https://github.com/pypa/pipx) to install `hatch`.

```bash
# install hatch in isolated env
pipx install hatch
# rebuild
hatch build
> [wheel]
> dist/remotivelabs_broker-<version>-py3-none-any.whl
# you can now install this library version in a virtualenv by opening a new terminal and doing:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip
pip3 install dist/remotivelabs_broker-<version>-py3-none-any.whl
# test that the package is importable
python -c "import remotivelabs.broker.sync as br; print(dir(br.SignalCreator))"
# when you are done, exit the virtualenv
deactivate
```

## Build and publish

Make sure to put version in `__about__.py`. Beta versions should be suffixed with `b*`, example `0.2.0b1`

```
./docker-build.sh
hatch publish
```
find username and password in less secret location
or use
```
hatch publish --user __token__
```
## License

`remotivelabs-broker` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.
7 changes: 1 addition & 6 deletions python/remotivelabs-broker/docker-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,4 @@ docker build -t "${TAG}" -f "${SCRIPT_DIR}/docker/Dockerfile" "${SCRIPT_DIR}"
ARGS="-it"
[ "${NO_TTY}" == "true" ] && ARGS="-i"

docker run \
-u "$(id -u):$(id -g)" \
-v "${SCRIPT_DIR}/../../:/app" \
-e "protofile=*.proto" \
-w /app \
"${ARGS}" "${TAG}"
docker run -v "${SCRIPT_DIR}/../../:/app" "${ARGS}" "${TAG}"
Loading

0 comments on commit a8c9b32

Please sign in to comment.