From b061ea31f08e73eb9f194deb12d119b311b3feb9 Mon Sep 17 00:00:00 2001 From: Richard Kuo Date: Sun, 24 Dec 2023 01:26:25 +0800 Subject: [PATCH] Delete _posts/2023-12-14-Reinforcement-Learning-for-Robotics.md --- ...-14-Reinforcement-Learning-for-Robotics.md | 317 ------------------ 1 file changed, 317 deletions(-) delete mode 100644 _posts/2023-12-14-Reinforcement-Learning-for-Robotics.md diff --git a/_posts/2023-12-14-Reinforcement-Learning-for-Robotics.md b/_posts/2023-12-14-Reinforcement-Learning-for-Robotics.md deleted file mode 100644 index 92c31475..00000000 --- a/_posts/2023-12-14-Reinforcement-Learning-for-Robotics.md +++ /dev/null @@ -1,317 +0,0 @@ ---- -layout: post -title: Reinforcement Learning for Robotics -author: [Richard Kuo] -category: [Lecture] -tags: [jekyll, ai] ---- - -Introduction to Reinforcement Learning for Robot / Drone. - ---- -## Embodied AI -**Blog:** [Overview of Embodied Artificial Intelligence](https://medium.com/machinevision/overview-of-embodied-artificial-intelligence-b7f19d18022)
-Embodied AI is the field for solving AI problems for virtual robots that can move, see, speak, and interact in the virtual world and with other virtual robots — these simulated robot solutions are then transferred to real world robots
-The simulated environments for Embodied AI training: SUNCG, Matterport3D, iGibson, Replica, Habitat, and DART
- ---- -### Matterport3D -**Paper:** [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://arxiv.org/abs/1709.06158)
-**Code:** [Matterport3D](https://github.com/niessner/Matterport)
- -![](https://github.com/niessner/Matterport/blob/master/img/teaser.jpg?raw=true) - ---- -### Replica -**Code:** [Replica Dataset](https://github.com/facebookresearch/Replica-Dataset)
-![](https://github.com/facebookresearch/Replica-Dataset/blob/main/assets/ReplicaModalities.png?raw=true) - ---- -### iGibson -**Paper:** [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://arxiv.org/abs/2108.03272)
-**Code:** [StanfordVL/iGibson](https://github.com/StanfordVL/iGibson)
-![](https://github.com/StanfordVL/iGibson/blob/master/docs/images/igibson.gif?raw=true) - ---- -### Habitat 2.0 -**Paper:** [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://arxiv.org/abs/2106.14405)
-**Code:** [facebookresearch/habitat-sim](https://github.com/facebookresearch/habitat-sim)
- - ---- -## Indoor Navigation - -### Autonomous Indoor Robot Navigation -**Paper:** [Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments](https://arxiv.org/abs/2005.13857)
-**Code:** [](https://github.com/RoblabWh/RobLearn)
- -![](https://d3i71xaburhd42.cloudfront.net/8a47843c2e664e5e7e218e2d891726d023619403/3-Figure4-1.png) - ---- -### DDPG 路徑規劃 -**Blog:** [智慧送餐服務型機器人導航路徑之設計](https://www.phdbooks.com.tw/cn/magazine/detail/1225)
-路徑跟隨器有四個主軸：
-* 送餐路徑生成：從文件或上層發佈訊息獲取預先定義的路徑。 -* 編輯航線路徑點：清除路徑中不合適的航線路徑點。 -* MFAC無模型自適應控制之航段管制：自動調整送餐路徑之導航點之間的航段長度，依序共分成路徑跟隨之依據以及MFAC無模型自適應控制之應用。 -* DWA之區域路徑傳遞：依照MFAC調整之結果，產出相關生成路徑，並以DWA進行區域設定。 - -* **自走車基於DDPG的室內路徑規劃**
- - ---- -### Long-Range Indoor Navigation -**Paper:** [Long-Range Indoor Navigation with PRM-RL](https://arxiv.org/abs/1902.09458)
- -![](https://github.com/rkuo2000/AI-course/blob/gh-pages/images/Long-Range-Indoor-Navigation.png?raw=true) - ---- -## Gym-Gazebo -**Code:** [erlerobot/gym-gazebo](https://github.com/erlerobot/gym-gazebo)
- - - - - - -
- ---- -## DART (Dynamic Animation and Robotics Toolkit) - -### [Dartsim/dart](https://github.com/dartsim/dart) -* Python bindings: dartpy, pydart2 (deprecated) -* OpenAI Gym with DART support: gym-dart (dartpy based), DartEnv (pydart2 based, deprecated) - - - ---- -## [PyBullet-Gym](https://github.com/benelot/pybullet-gym) -[PyBullet Quickstart Guide](https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.2ye70wns7io3)
- -**code:** [rkuo2000/pybullet-gym](https://github.com/rkuo2000/pybullet-gym)
-* installation -``` -pip install gym -pip install stable-baselines3 -git clone https://github.com/rkuo2000/pybullet-gym -export PYTHONPATH=$PATH:/home/yourname/pybullet-gym -``` - -**Train**
-`python train.py Ant 10000000`
- -**Enjoy** with trained-model
-`python enjoy.py Ant`
- -**Enjoy** with pretrained weights
-`python enjoy_Ant.py`
-`python enjoy_HumanoidFlagrunHarder.py` (a copy from pybulletgym/examples/roboschool-weights/enjoy_TF_*.py)
- ---- -### [PyBullet-Robots](https://github.com/erwincoumans/pybullet_robots) - - -**env_name = "AtlasPyBulletEnv-v0"**
-[atlas_v4_with_multisense.urdf](https://github.com/benelot/pybullet-gym/blob/master/pybulletgym/envs/assets/robots/atlas/atlas_description/atlas_v4_with_multisense.urdf)
- - ---- -## RoboCar Gym - -### Pybullet-RoboCar -**Blog:**
-[Creating OpenAI Gym Environments with PyBullet (Part 1)](https://gerardmaggiolino.medium.com/creating-openai-gym-environments-with-pybullet-part-1-13895a622b24)
-[Creating OpenAI Gym Environments with PyBullet (Part 2)](https://gerardmaggiolino.medium.com/creating-openai-gym-environments-with-pybullet-part-2-a1441b9a4d8e)
-![](https://media0.giphy.com/media/VI3OuvQShK3gzENiVz/giphy.gif?cid=790b761131bda06b74fcd9bb06c6a43939cf446edf403a68&rid=giphy.gif&ct=g) - ---- -## Quadruped Gym - -### [Motion Imitation](https://github.com/google-research/motion_imitation) -**Code:** [TF 1.15](https://github.com/google-research/motion_imitation)
- -**Code:** [PyTorch](https://github.com/newera-001/motor-system)
-For Training:
-`python motion_imitation/run_torch.py --mode train --motion_file 'dog_pace.txt|dog_spin.txt' \ ---int_save_freq 10000000 --visualize --num_envs 50 --type_name 'dog_pace'`
-For Testing:
-`python motion_imitation/run_torch.py --mode test --motion_file 'dog_pace.txt' --model_file 'file_path' \ ---encoder_file 'file_path' --visualize`
- ---- -### Rex: an open-source quadruped robot -**Code:** [nicrusso7/rex-gym](https://github.com/nicrusso7/rex-gym)
-![](https://github.com/nicrusso7/rex-gym/blob/master/images/intro.gif?raw=true) - ---- -## Drones Gym - -### [PyBullet-Gym for Drones](https://github.com/utiasDSL/gym-pybullet-drones) -![](https://github.com/utiasDSL/gym-pybullet-drones/blob/master/files/readme_images/helix.gif?raw=true) -![](https://github.com/utiasDSL/gym-pybullet-drones/blob/master/files/readme_images/helix.png?raw=true) - -* Installation -``` -sudo apt install ffmpeg -pip install numpy pillow matplotlib cycler -pip install gym pybullet stable_baselines3 ray[rllib] -git clone https://github.com/rkuo2000/gym-pybullet-drones.git -cd gym-pybullet-drones -``` - -* Train & Enjoy
-`python train.py` # modify train.py for different env, algorithm and timesteps
-`python enjoy.py` # modify enjoy.py for different env
- -* Fly using [DSLPIDControl.py](https://github.com/utiasDSL/gym-pybullet-drones/blob/master/gym_pybullet_drones/control/DSLPIDControl.py):（PID飛行）
-`python examples/fly.py --num_drones 1`
-![](https://github.com/utiasDSL/gym-pybullet-drones/blob/master/files/readme_images/wp.gif?raw=true) - -* To learn take-off:（起飛）
-`python examples/learn.py`
-![](https://github.com/utiasDSL/gym-pybullet-drones/blob/master/files/readme_images/learn2.gif?raw=true) - -* `compare.py` which replays and compare to a trace saved in `files/example_trace.pkl` - -**Experiments**
-`cd experiments/learning`
- -env : hover, takeoff, flythrugate, tune（旋停, 起飛, 穿越, 調整）
-algo: a2c, ppo, sac, td3, ddpg
- -* To learn hover:（旋停）
-`python singleagent.py --env hover --algo a2c`
- -To visualize the best trained agent:
-`python test_singleagent.py --exp ./results/save-hover-a2c`
- -For multi-agent RL, using rllib:
-`python multiagent.py --num_drones 3 --env hover --algo a2c --num_workers 2`
- ---- -### [Flightmare](https://github.com/uzh-rpg/flightmare) -Flightmare is a flexible modular quadrotor simulator. - - -* [Introduction](https://github.com/uzh-rpg/flightmare/wiki/Introduction) -* [Prerequisites](https://github.com/uzh-rpg/flightmare/wiki/Prerequisites) -* [Install Python Packages](https://github.com/uzh-rpg/flightmare/wiki/Install-with-pip) -* [Install ROS](https://github.com/uzh-rpg/flightmare/wiki/Install-with-ROS) - -* **running ROS** -``` -roslaunch flightros rotors_gazebo.launch -``` - -* **flighRL**
-``` -cd /path/to/flightmare/flightrl -pip install . -cd examples -python3 run_drone_control.py --train 1 -``` - ---- -### [AirSim](https://github.com/microsoft/AirSim) - -![](https://github.com/microsoft/AirSim/blob/master/docs/images/AirSimDroneManual.gif?raw=true) - ---- -## Assistive Gym - -**Paper:** [Assistive Gym: A Physics Simulation Framework for Assistive Robotics](https://arxiv.org/abs/1910.04700)
-![](https://github.com/Healthcare-Robotics/assistive-gym/blob/main/images/assistive_gym.jpg?raw=true) - - -* Four collaborative robots (PR2, Jaco, Baxter, Sawyer)
-![](https://github.com/Healthcare-Robotics/assistive-gym/raw/main/images/robot_models.gif) - -* Support for the Stretch and PANDA robots
- - - - - -
- -**Code:** [Healthcare-Robotics/assistive-gym](https://github.com/Healthcare-Robotics/assistive-gym)
- ---- -### Assistive VR Gym -**Paper:** [Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots](https://arxiv.org/abs/2007.04959)
-**Code:** [Healthcare-Robotics/assistive-vr-gym](https://github.com/Healthcare-Robotics/assistive-vr-gym)
-![](https://github.com/Healthcare-Robotics/assistive-vr-gym/blob/master/images/avr_gym_2.jpg?raw=true) - - ---- -## Learning Dexity - - ---- -### [Dexterous Gym](https://github.com/henrycharlesworth/dexterous-gym) -![](https://github.com/henrycharlesworth/dexterous-gym/raw/master/dexterous_gym/examples/penspin.gif) -![](https://github.com/henrycharlesworth/dexterous-gym/raw/master/dexterous_gym/examples/egghandover.gif) - ---- -### [DexPilot](https://research.nvidia.com/publication/2020-05_dexpilot-vision-based-teleoperation-dexterous-robotic-hand-arm-system) -**Paper:** [DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System](https://arxiv.org/abs/1910.03135)
-![](https://research.nvidia.com/sites/default/files/styles/wide/public/publications/dexpilot.jpg?itok=7Re04wXI) - - ---- -### [TriFinger](https://sites.google.com/view/trifinger) - - -**Paper:** [TriFinger: An Open-Source Robot for Learning Dexterity](https://arxiv.org/abs/ -2008.03596)
-![](https://ai2-s2-public.s3.amazonaws.com/figures/2017-08-08/f223b2a438fe20ac55300c278509bf4a13072ca8/3-Figure2-1.png) -![](https://ai2-s2-public.s3.amazonaws.com/figures/2017-08-08/f223b2a438fe20ac55300c278509bf4a13072ca8/4-Figure3-1.png) -![](https://ai2-s2-public.s3.amazonaws.com/figures/2017-08-08/f223b2a438fe20ac55300c278509bf4a13072ca8/4-Figure4-1.png) - -**Code:** [TriFinger Robot Simulation](https://github.com/open-dynamic-robot-initiative/trifinger_simulation)
- - ---- -### [Multi-Task Reset-Free (MTRF) Learning](https://sites.google.com/view/mtrf) -**Paper:** [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://arxiv.org/abs/2104.11203)
- - ---- -### Dexterous Anthropomorphic Robotic Hand -**Blog:** [Robotic hand can crush beer cans and hold eggs without breaking them](https://www.newscientist.com/article/2301641-robotic-hand-can-crush-beer-cans-and-hold-eggs-without-breaking-them/)
-![](https://images.newscientist.com/wp-content/uploads/2021/12/14124151/PRI_215070295.jpg?width=778) - -**Paper:** [Integrated linkage-driven dexterous anthropomorphic robotic hand](https://www.nature.com/articles/s41467-021-27261-0#Abs1)
-![](https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41467-021-27261-0/MediaObjects/41467_2021_27261_Fig2_HTML.png?as=webp) - - - ---- -### Robotic Telekinesis -[Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube](https://arxiv.org/abs/2202.10448)
- - ---- -### Fixed-Finger Gripper -**Paper:** [F1 Hand: A Versatile Fixed-Finger Gripper for Delicate Teleoperation and Autonomous Grasping](https://arxiv.org/abs/2205.07066)
- - ---- -### Learning Diverse Dexterous Grasps -**Paper:** [Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization](https://arxiv.org/abs/2207.00195)
- - ---- -### ViLa -**Blog:** [https://bangqu.com/9Fa2ra.html](https://bangqu.com/9Fa2ra.html)
-![](https://i3.res.bangqu.com/farm/j/news/2023/12/12/42e2981b43a54c3ac7a8647515a0ecaa.gif) - -
-
- -*This site was last updated {{ site.time | date: "%B %d, %Y" }}.* - -