Basic arm reaching environment created, myoArmReachRandom-v0

[jamesheald]

I have created a basic environment for reaching to random targets in space with the MyoSuite arm.

• I created a new myoarm_reach.xml file by modifying the existing myoarm_relocate.xml file (I removed the object, table and bin and added a target as a site element).
• I used the ReachEnvV0 environment class as the entry point for the environment.
• The palm position ('S_grasp') is the part of the arm that needs to reach the target.
• I registered a new environment, ''myoArmReachRandom-v0', in /myosuite/myosuite/envs/myo/myobase/init.py.
• The target_reach_range has been set to encourage reasonable exploration of the 3D space.
• Let me know if you have an established methodology for determining the ideal environment parameters (e.g. target_reach_range, max_episode_steps, far_th). I'd love to optimise these so that the environment can be made available to the community (a couple of people have already messaged me in the MyoSuite slack workspace to ask if they can access the environment).

[vikashplus]

Looks good to me. Next step will be train a policy to ensure all parameters are playing well.

[jamesheald]

Following Vittorio's advice, I have created a myoArm model that has no hand or hand muscles (see myo_sim PR 29).

I have created a fixed and random variant of the task:

1. Fixed: single target
2. Random: random targets in a 30cm x 30cm x 30cm cube

Results of training for the fixed and random variants of the environment look good to me.

[jamesheald]

Vittorio suggested to include the hand bones and joints (previously excluded) and only exclude the hand tendons and muscles. I've done this and trained a SAC policy on the random variant of the task. Here are the training results.