Skip to content

Commit

Permalink
Automate PyPi publusing using CI
Browse files Browse the repository at this point in the history
  • Loading branch information
rodrigo-ceccato committed Nov 7, 2024
1 parent 8397c37 commit c4ad74e
Show file tree
Hide file tree
Showing 3 changed files with 207 additions and 11 deletions.
36 changes: 36 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Publish Package to PyPI

on:
push:
tags:
- 'v*.*.*' # Matches tags like v1.0.0

jobs:
publish:
runs-on: ubuntu-latest

steps:
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0 # Fetch all history for all branches and tags

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build package
run: python -m build

- name: Publish package
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: twine upload dist/*

170 changes: 160 additions & 10 deletions docs/contribute.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,57 @@
# Contribute to Spinner

## Setting up the environment as a developer
Welcome to the Spinner project! This guide will help you set up your development environment, understand our linting policies, learn how to use Git tags for versioning, and understand how our build automation works.

## Table of Contents

- [Setting Up the Development Environment](#setting-up-the-development-environment)
- [Mandatory Lint Policy](#mandatory-lint-policy)
- [Versioning with Git Tags](#versioning-with-git-tags)
- [Using Git Tags](#using-git-tags)
- [Best Practices for Tagging](#best-practices-for-tagging)
- [Build Automation with GitHub Actions](#build-automation-with-github-actions)
- [How the Build Bot Works](#how-the-build-bot-works)
- [Configuring PyPI Credentials](#configuring-pypi-credentials)
- [Versioning with `setuptools_scm`](#versioning-with-setuptools_scm)
- [Conclusion](#conclusion)

## Setting Up the Development Environment

To set up your development environment for Spinner, follow these steps:

```sh
python3 -m ensurepip
python3 -m pip3 install virtualenv
python3 -m pip install virtualenv
python3 -m virtualenv .venv
source .venv/bin/activate
python3 -m pip install pip --upgrade
python -m pip install -e ".[dev]"
```

## Mandatory lint policy
This will:

- **Ensure `pip` is installed**.
- **Install `virtualenv`** if it's not already installed.
- **Create a virtual environment** in the `.venv` directory.
- **Activate the virtual environment**.
- **Upgrade `pip`** to the latest version.
- **Install the package** in editable mode along with development dependencies specified in `pyproject.toml`.

## Mandatory Lint Policy

We enforce a mandatory linting policy to maintain code quality and consistency. We recommend adding the following pre-commit hook to automatically format your code before each commit.

### Setting Up the Pre-Commit Hook

Create a file named `pre-commit` in the `.git/hooks/` directory and make it executable:

```sh
touch .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
```

Add the following content to the `pre-commit` file:

We recomend the following commit hook (`.git/hooks/pre-commit`)
```sh
#!/bin/bash

Expand All @@ -36,25 +74,29 @@ format_c_cpp_files() {
fi
}

# Function to format all Python files using black
# Function to format all Python files using black and isort
format_python_files() {
if command -v black >/dev/null 2>&1; then
if command -v black >/dev/null 2>&1 && command -v isort >/dev/null 2>&1; then
for file in $(git diff --cached --name-only --diff-filter=ACM "$against" | grep -E '\.py$'); do
black "$file"
isort "$file"
git add "$file"
done
else
echo "black not found, skipping Python file formatting."
echo "black or isort not found, skipping Python file formatting."
fi
if command -v black >/dev/null 2>&1; then
}

# Function to format Jupyter notebooks
format_notebooks() {
if command -v black >/dev/null 2>&1 && command -v isort >/dev/null 2>&1; then
for file in $(git diff --cached --name-only --diff-filter=ACM "$against" | grep -E '\.ipynb$'); do
black "$file"
isort "$file"
git add "$file"
done
else
echo "black for notebook not found, skipping Python file formatting."
echo "black or isort not found, skipping notebook formatting."
fi
}

Expand All @@ -73,8 +115,116 @@ clear_notebook_output() {
# Run the formatting functions
format_c_cpp_files
format_python_files
format_notebooks
clear_notebook_output

# Check for changes
exec git diff-index --check --cached "$against" --
```
```

This script will:

- **Format C/C++ files** using `clang-format`.
- **Format Python files and notebooks** using `black` and `isort`.
- **Clear outputs** from Jupyter notebooks to avoid committing unnecessary data.
- **Automatically add the formatted files** back to the commit.

## Versioning with Git Tags

We use Git tags to manage our release versions, which are crucial for packaging and distributing the Spinner project via PyPI.

### Using Git Tags

To create a new Git tag for a release, follow these steps:

1. **Ensure all changes are committed**:

```sh
git add .
git commit -m "Prepare for release v0.0.2"
```

2. **Create a new tag**:

```sh
git tag v0.0.2
```

Replace `v0.0.2` with the appropriate version number.

3. **Push the tag to GitHub**:

```sh
git push origin v0.0.2
```

### Best Practices for Tagging

- **Semantic Versioning**: We follow [Semantic Versioning](https://semver.org/) with the format `vMAJOR.MINOR.PATCH` (e.g., `v1.2.3`).
- **MAJOR** version when you make incompatible API changes.
- **MINOR** version when you add functionality in a backwards-compatible manner.
- **PATCH** version when you make backwards-compatible bug fixes.

- **Annotated Tags**: Use annotated tags to include additional metadata such as the tagger name, email, date, and a message.

```sh
git tag -a v0.0.2 -m "Release version 0.0.2"
```

- **Consistency**: Always start tags with a `v` to denote version (e.g., `v0.0.2`).

- **Changelog Updates**: Before tagging, update the `CHANGELOG.md` to reflect the changes in the new version.

- **Testing Before Release**: Ensure all tests pass before creating a tag.

## Build Automation with GitHub Actions

Our project uses GitHub Actions to automate the build and deployment process. When a new tag is pushed to the repository, the build bot (GitHub Actions workflow) is triggered to:

- **Build the Package**: Create source and wheel distributions of the package.
- **Publish to PyPI**: Upload the distributions to PyPI using Twine.

### How the Build Bot Works

1. **Trigger**: The workflow is triggered on pushing tags that match the pattern `v*.*.*` (e.g., `v0.0.2`).

2. **Workflow File**: The workflow is defined in `.github/workflows/publish.yml`.

3. **Steps**:
- **Checkout Code**: Fetches the repository code and tags.
- **Set Up Python**: Configures the Python environment.
- **Install Dependencies**: Installs build tools like `build` and `twine`.
- **Build Package**: Uses `python -m build` to create distributions.
- **Publish Package**: Uploads the package to PyPI using Twine and the stored PyPI API token.

### Configuring PyPI Credentials

- **PyPI API Token**: The API token for PyPI is stored securely in GitHub Secrets under the name `PYPI_API_TOKEN`.
- **Security**: Never commit API tokens or passwords to the repository.

### Versioning with `setuptools_scm`

We use `setuptools_scm` to automatically manage the package version based on Git tags.

- **Dynamic Versioning**: The version number is derived from the latest Git tag.
- **Configuration**: Ensure `pyproject.toml` is set up correctly with `dynamic = ["version"]` and includes `setuptools_scm` in the build requirements.

### Important Notes

- **Ensure Tags are Pushed**: The build bot relies on the Git tags to determine the version. Always push your tags to GitHub.
- **Check Workflow Status**: After pushing a tag, monitor the Actions tab in GitHub to ensure the workflow runs successfully.
- **Test Locally**: Before pushing a tag, you can test the build process locally:

```sh
python -m build
```

## Conclusion

By following these guidelines, you can contribute effectively to the Spinner project. Our versioning and build automation processes ensure that releases are consistent and that our package is reliably published to PyPI.

If you have any questions or need assistance, feel free to reach out to the maintainers.

---

Happy coding!
12 changes: 11 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ authors = [
{name = "Rodrigo Ceccato de Freitas", email = "[email protected]"},
{name = "Jhonatan Cléto", email="[email protected]"}
]
version = "0.0.1"

dynamic = ["version"]

requires-python = ">=3.9.12, <4"
dependencies = [
"click==8.1.7",
Expand Down Expand Up @@ -49,3 +51,11 @@ line_length = 79
[tool.taskipy.tasks]
lint = "black --check --diff . && isort --check --diff ."
format = "black . && isort ."

[build-system]
requires = ["setuptools>=42", "wheel", "setuptools_scm"]
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
version_scheme = "release-branch-semver"
local_scheme = "no-local-version"

0 comments on commit c4ad74e

Please sign in to comment.