From 6dc0e016321435c93cafcc71557db4c8f574ecac Mon Sep 17 00:00:00 2001 From: Amit Parekh <7276308+amitkparekh@users.noreply.github.com> Date: Sun, 3 Dec 2023 05:44:50 +0000 Subject: [PATCH] docs: improve the readme --- README.md | 131 +++++------------------------------------------------- 1 file changed, 10 insertions(+), 121 deletions(-) diff --git a/README.md b/README.md index 2840726..9535bc9 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@
-# EMMA SimBot Research Base +# EMMA: Perception Python 3.9 @@ -35,11 +35,17 @@ [![Continuous Integration](https://github.com/emma-heriot-watt/perception/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/continuous_integration.yml) [![Tests](https://github.com/emma-heriot-watt/perception/actions/workflows/tests.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/tests.yml) +[![Build and push images](https://github.com/emma-heriot-watt/perception/actions/workflows/build-image.yml/badge.svg)](https://github.com/emma-heriot-watt/perception/actions/workflows/build-image.yml)
--- +The object detector and feature extractor used for the EMMA project. + + + + A clean and scalable [PyTorch Lightning and Hydra Template](https://github.com/ashleve/lightning-hydra-template) for deep learning research, with the simplicity of [Hypermodern Python](https://github.com/cjolowicz/cookiecutter-hypermodern-python) tooling. Assuming you have [pyenv](https://github.com/pyenv/pyenv) and [Poetry](https://python-poetry.org/), clone the repository and run: @@ -118,64 +124,11 @@ This is organised in very similarly to structure from the [Lightning-Hydra-Templ - `scripts` — `sh` scripts to run experiments - `configs` — configurations files using the [Hydra framework](https://hydra.cc/) -- `docker` — Dockerfiles to ease deployment - `notebooks` — Jupyter notebook for analysis and exploration - `storage` — data for training/inference _(and maybe use symlinks to point to other parts of the filesystem)_ -- `tests` — [pytest](https://docs.pytest.org/en/) scripts to verify the code - `src` — where the main code lives -## Running things - -Train model with default configuration - -```bash -# train on CPU -python run.py trainer.gpus=0 - -# train on 1 GPU -python run.py trainer.gpus=1 -``` - -Train model with chosen experiment configuration from [configs/experiment/](configs/experiment/) - -```bash -python run.py experiment=experiment_name.yaml -``` - -You can override any parameter from command line like this -```bash -python run.py trainer.max_epochs=20 datamodule.batch_size=64 -``` - -Especially when you're trying to specify extra parameters for the class `Trainer` from -[PyTorch-Lightning](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.trainer.html#module-pytorch_lightning.trainer.trainer), -you might run into trouble when running a command like the following: - -```bash -python run.py trainer.precision=16 datamodule.batch_size=64 -``` - -This is because Hydra allows you to modify only parameters specified in the configuration file. So -if you don't have `precision` among them, Hydra will complain. If you're sure that the parameters -is allowed, just change the previous command as follows: - -```bash -python run.py +trainer.precision=16 datamodule.batch_size=64 -``` - -In this way, Hydra will automatically _append_ the new parameter to the configuration dictionary of -the `Trainer` we're trying to instantiate. - -
-It's annoying, why do I have to do that? - -
- -We're working on a possible fix and we're exploring different options. If you're interested in -this, please follow this [issue](https://github.com/emma-simbot/research-base/issues/26). - -
## Developer tooling @@ -186,74 +139,10 @@ this, please follow this [issue](https://github.com/emma-simbot/research-base/is - Automated Python Docstring Formatting with [docformatter](https://github.com/myint/docformatter) - Continuous integration with [GitHub Actions](https://github.com/features/actions) - Testing with [pytest](https://docs.pytest.org/en/latest/) -- Code coverage with [coverage.oy](https://coverage.readthedocs.io/) +- Code coverage with [coverage.py](https://coverage.readthedocs.io/) - Static type-checking with [mypy](http://mypy-lang.org/) - Automated Python syntax updates with [pyupgrade](https://github.com/asottile/pyupgrade) - Security audit with [Bandit](https://github.com/PyCQA/bandit) -- Automated release notes with [Release Drafter](https://github.com/release-drafter/release-drafter) - Manage project labels with [GitHub Labeler](https://github.com/marketplace/actions/github-labeler) -- Automated dependency updates with [Dependabot](https://dependabot.com/) - -### Code style - -To ensure all code is standardized, we use [black](https://github.com/psf/black), along with other automatic formatters. To enforce a consistent coding style, we use [Flake8](https://flake8.pycqa.org/en/latest/) with the [wemake-python-styleguide](https://wemake-python-stylegui.de/en/latest/). To verify and enforce type annotations, we use [mypy](https://mypy.readthedocs.io/en/stable/). Common tasks can be called using [Poe the Poet](https://github.com/nat-n/poethepoet). - -#### Task runner - -[Poe the Poet](https://github.com/nat-n/poethepoet) is a task runner that works well with Poetry. To see what tasks exist, run `poe` in your terminal. If you want to add more tasks, check [the documentation](https://github.com/nat-n/poethepoet) and [what already exists](https://github.com/emma-simbot/research-base/blob/main/pyproject.toml). - -If you have issues getting `poe` to work, make sure that you are already within the activated shell (by running `poetry shell`). - -#### Formatting - -If you want to automatically format on every commit, you can use [pre-commit](https://pre-commit.com/). As mentioned above, run `pre-commit install` and it will install the hooks. - -To manually format all files, run - -```bash -poe format -``` - -#### Linting - -If you want to check the linting rules on the codebase, run - -```bash -poe lint -``` - -#### Type checking - -If you want to check types on the codebase, run - -```bash -poe typecheck -``` - -#### Working with branches - -We've settled on a middle ground when it comes to developing: **keep the `main` branch clean**. - -Within branches, you can do whatever you want to do, but you should **never push anything directly to the `main` branch**. - -For every PR, an automated set of linters and formatters will check your code to see whether it follows the set rules. If it fails, **do not merge** with the `main` branch. - -##### Before submitting - -1. Run `pytest` to make sure no errors before you start. -2. Add any changes you want. -3. Add tests for changes -4. Add integration test -5. Run `pytest` again -6. Run `mypy` - -_NOTE: This list needs updating._ - -## Other repositories - -### Inspirations - -- [ashleve/Lightning-Hydra-Template](https://github.com/ashleve/lightning-hydra-template) -- [cjolowicz/cookiecutter-hypermodern-python](https://github.com/cjolowicz/cookiecutter-hypermodern-python) -- [RasaHQ/rasa](https://github.com/RasaHQ/rasa) -- [wemake-services/wemake-python-stylguide](https://github.com/wemake-services/wemake-python-styleguide) +- ~~Automated release notes with [Release Drafter](https://github.com/release-drafter/release-drafter)~~ +- ~~Automated dependency updates with [Dependabot](https://dependabot.com/)~~