Skip to content

Commit

Permalink
docs: improve the readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@amitkparekh
amitkparekh committed Dec 3, 2023
1 parent 9f6b83f commit 6dc0e01
Showing 1 changed file with 10 additions and 121 deletions.
131 changes: 10 additions & 121 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<div align='center'>

# EMMA SimBot Research Base
# EMMA: Perception

<a href="https://www.python.org/">
<img alt="Python 3.9" src="https://img.shields.io/badge/-Python 3.9+-blue?logo=python&logoColor=white">
Expand Down Expand Up @@ -35,11 +35,17 @@

[![Continuous Integration](https://github.com/emma-heriot-watt/perception/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/continuous_integration.yml)
[![Tests](https://github.com/emma-heriot-watt/perception/actions/workflows/tests.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/tests.yml)
[![Build and push images](https://github.com/emma-heriot-watt/perception/actions/workflows/build-image.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/build-image.yml)

</div>

---

The object detector and feature extractor used for the EMMA project.




A clean and scalable [PyTorch Lightning and Hydra Template](https://github.com/ashleve/lightning-hydra-template) for deep learning research, with the simplicity of [Hypermodern Python](https://github.com/cjolowicz/cookiecutter-hypermodern-python) tooling.

Assuming you have [pyenv](https://github.com/pyenv/pyenv) and [Poetry](https://python-poetry.org/), clone the repository and run:
Expand Down Expand Up @@ -118,64 +124,11 @@ This is organised in very similarly to structure from the [Lightning-Hydra-Templ

- `scripts` — `sh` scripts to run experiments
- `configs` — configurations files using the [Hydra framework](https://hydra.cc/)
- `docker` — Dockerfiles to ease deployment
- `notebooks` — Jupyter notebook for analysis and exploration
- `storage` — data for training/inference _(and maybe use symlinks to point to other parts of the filesystem)_
- `tests` — [pytest](https://docs.pytest.org/en/) scripts to verify the code
- `src` — where the main code lives

## Running things

Train model with default configuration

```bash
# train on CPU
python run.py trainer.gpus=0

# train on 1 GPU
python run.py trainer.gpus=1
```

Train model with chosen experiment configuration from [configs/experiment/](configs/experiment/)

```bash
python run.py experiment=experiment_name.yaml
```

You can override any parameter from command line like this

```bash
python run.py trainer.max_epochs=20 datamodule.batch_size=64
```

Especially when you're trying to specify extra parameters for the class `Trainer` from
[PyTorch-Lightning](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.trainer.html#module-pytorch_lightning.trainer.trainer),
you might run into trouble when running a command like the following:

```bash
python run.py trainer.precision=16 datamodule.batch_size=64
```

This is because Hydra allows you to modify only parameters specified in the configuration file. So
if you don't have `precision` among them, Hydra will complain. If you're sure that the parameters
is allowed, just change the previous command as follows:

```bash
python run.py +trainer.precision=16 datamodule.batch_size=64
```

In this way, Hydra will automatically _append_ the new parameter to the configuration dictionary of
the `Trainer` we're trying to instantiate.

<details>
<summary>It's annoying, why do I have to do that? </summary>

<br>

We're working on a possible fix and we're exploring different options. If you're interested in
this, please follow this [issue](https://github.com/emma-simbot/research-base/issues/26).

</details>

## Developer tooling

Expand All @@ -186,74 +139,10 @@ this, please follow this [issue](https://github.com/emma-simbot/research-base/is
- Automated Python Docstring Formatting with [docformatter](https://github.com/myint/docformatter)
- Continuous integration with [GitHub Actions](https://github.com/features/actions)
- Testing with [pytest](https://docs.pytest.org/en/latest/)
- Code coverage with [coverage.oy](https://coverage.readthedocs.io/)
- Code coverage with [coverage.py](https://coverage.readthedocs.io/)
- Static type-checking with [mypy](http://mypy-lang.org/)
- Automated Python syntax updates with [pyupgrade](https://github.com/asottile/pyupgrade)
- Security audit with [Bandit](https://github.com/PyCQA/bandit)
- Automated release notes with [Release Drafter](https://github.com/release-drafter/release-drafter)
- Manage project labels with [GitHub Labeler](https://github.com/marketplace/actions/github-labeler)
- Automated dependency updates with [Dependabot](https://dependabot.com/)

### Code style

To ensure all code is standardized, we use [black](https://github.com/psf/black), along with other automatic formatters. To enforce a consistent coding style, we use [Flake8](https://flake8.pycqa.org/en/latest/) with the [wemake-python-styleguide](https://wemake-python-stylegui.de/en/latest/). To verify and enforce type annotations, we use [mypy](https://mypy.readthedocs.io/en/stable/). Common tasks can be called using [Poe the Poet](https://github.com/nat-n/poethepoet).

#### Task runner

[Poe the Poet](https://github.com/nat-n/poethepoet) is a task runner that works well with Poetry. To see what tasks exist, run `poe` in your terminal. If you want to add more tasks, check [the documentation](https://github.com/nat-n/poethepoet) and [what already exists](https://github.com/emma-simbot/research-base/blob/main/pyproject.toml).

If you have issues getting `poe` to work, make sure that you are already within the activated shell (by running `poetry shell`).

#### Formatting

If you want to automatically format on every commit, you can use [pre-commit](https://pre-commit.com/). As mentioned above, run `pre-commit install` and it will install the hooks.

To manually format all files, run

```bash
poe format
```

#### Linting

If you want to check the linting rules on the codebase, run

```bash
poe lint
```

#### Type checking

If you want to check types on the codebase, run

```bash
poe typecheck
```

#### Working with branches

We've settled on a middle ground when it comes to developing: **keep the `main` branch clean**.

Within branches, you can do whatever you want to do, but you should **never push anything directly to the `main` branch**.

For every PR, an automated set of linters and formatters will check your code to see whether it follows the set rules. If it fails, **do not merge** with the `main` branch.

##### Before submitting

1. Run `pytest` to make sure no errors before you start.
2. Add any changes you want.
3. Add tests for changes
4. Add integration test
5. Run `pytest` again
6. Run `mypy`

_NOTE: This list needs updating._

## Other repositories

### Inspirations

- [ashleve/Lightning-Hydra-Template](https://github.com/ashleve/lightning-hydra-template)
- [cjolowicz/cookiecutter-hypermodern-python](https://github.com/cjolowicz/cookiecutter-hypermodern-python)
- [RasaHQ/rasa](https://github.com/RasaHQ/rasa)
- [wemake-services/wemake-python-stylguide](https://github.com/wemake-services/wemake-python-styleguide)
- ~~Automated release notes with [Release Drafter](https://github.com/release-drafter/release-drafter)~~
- ~~Automated dependency updates with [Dependabot](https://dependabot.com/)~~

0 comments on commit 6dc0e01

Please sign in to comment.