README.md

multi-task-pybullet-drones

This repository builds off the work done in gym-pybullet-drones with extensions focused on exploring multi-task learning.

Key changes include a YAML configuration system for rapidly testing environment designs, rewards, and terminations. New environments can be found in the env directory. These use configurations of the reward and terminations that can be found in rewards.

Simple OpenAI Gym environment based on PyBullet for multi-agent reinforcement learning with quadrotors

• The default DroneModel.CF2X dynamics are based on Bitcraze's Crazyflie 2.x nano-quadrotor

Requirements and Installation

The repo was written using Python 3.7 with conda on macOS 10.15 and tested with Python 3.8 on macOS 12, Ubuntu 20.04

On macOS and Ubuntu

Major dependencies are gym, pybullet, stable-baselines3, and rllib

Video recording requires to have ffmpeg installed, on macOS

$ brew install ffmpeg

On Ubuntu

$ sudo apt install ffmpeg

macOS with Apple Silicon (like the M1 Air) can only install grpc with a minimum Python version of 3.9 and these two environment variables set:

$ export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
$ export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

The repo is structured as a Gym Environment and can be installed with pip install --editable

$ conda create -n drones python=3.8 # or 3.9 on Apple Silicon, see the comment on grpc above
$ conda activate drones
$ pip3 install --upgrade pip
$ git clone https://github.com/utiasDSL/gym-pybullet-drones.git
$ cd gym-pybullet-drones/
$ pip3 install -e .

On Ubuntu and with a GPU available, optionally uncomment line 203 of BaseAviary.py to use the eglPlugin

gym-pybullet-drones Citation

If you wish, please cite our work (link) as

@INPROCEEDINGS{panerati2021learning,
      title={Learning to Fly---a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control}, 
      author={Jacopo Panerati and Hehui Zheng and SiQi Zhou and James Xu and Amanda Prorok and Angela P. Schoellig},
      booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      year={2021},
      volume={},
      number={},
      pages={},
      doi={}
}

References

• Nathan Michael, Daniel Mellinger, Quentin Lindsey, Vijay Kumar (2010) The GRASP Multiple Micro UAV Testbed
• Benoit Landry (2014) Planning and Control for Quadrotor Flight through Cluttered Environments
• Julian Forster (2015) System Identification of the Crazyflie 2.0 Nano Quadrocopter
• Carlos Luis and Jeroome Le Ny (2016) Design of a Trajectory Tracking Controller for a Nanoquadcopter
• Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor (2017) AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
• Eric Liang, Richard Liaw, Philipp Moritz, Robert Nishihara, Roy Fox, Ken Goldberg, Joseph E. Gonzalez, Michael I. Jordan, and Ion Stoica (2018) RLlib: Abstractions for Distributed Reinforcement Learning
• Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann (2019) Stable Baselines3
• Guanya Shi, Xichen Shi, Michael O’Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung (2019) Neural Lander: Stable Drone Landing Control Using Learned Dynamics
• Mikayel Samvelyan, Tabish Rashid, Christian Schroeder de Witt, Gregory Farquhar, Nantas Nardelli, Tim G. J. Rudner, Chia-Man Hung, Philip H. S. Torr, Jakob Foerster, and Shimon Whiteson (2019) The StarCraft Multi-Agent Challenge
• C. Karen Liu and Dan Negrut (2020) The Role of Physics-Based Simulators in Robotics
• Yunlong Song, Selim Naji, Elia Kaufmann, Antonio Loquercio, and Davide Scaramuzza (2020) Flightmare: A Flexible Quadrotor Simulator

Bonus GIF for scrolling this far

---

University of Toronto's Dynamic Systems Lab / Vector Institute / University of Cambridge's Prorok Lab / Mitacs