Implementation of the classic cart pole environment using standard Simulink blocks to model the underlying differential equations of the dynamics.
The environment is modelled in cartpole_simulink.slx and wrapped in cartpole_simulink.py.
Like the Gym implementation, this model uses a fixed step-size of 0.02 s and the Euler integration method (ode1 (Euler) solver).
Like the Gym implementation, this environment has a discrete action space with two actions allowing the agent to push the cart in both directions with a force of 10 N.
The observation space consists of the cart's position and velocity as well as the pole's angle and its angular velocity.
The dynamics of the system are defined by the physical properties of the cart and pole. These parameters are defined in the model workspace. The step size of the solver is also setup as a workspace variable. The following table lists all workspace variables and their default value. In the case of the initial cart position and pole angle the default values will be overwritten by the Simulink Gym wrapper.
| Property | Variable Name | Default Value | Unit |
|---|---|---|---|
| earth's gravity | g |
9.80665 | m/s² |
| length of pole (distance of joint to CoM) | length_pole |
0.5 | m |
| mass of pole | mass_pole |
0.1 | kg |
| mass of cart | mass_cart |
1 | kg |
| initial cart position | x_0 |
0 | m |
| initial pole angle | theta_0 |
° | |
| step size | step_size |
0.02 | s |
Check out the notebook to play around with the environment.
Also included in this directory are two example scripts for training a DQN (train_dqn_cartpole.py) and a PPO (train_ppo_cartpole.py) agent on the cart pole environment implemented in Simulink. The scripts use Stable-Baselines3 for the RL algorithms and, optionally, Weights & Biases for experiment tracking. For additional information on the usage of the example scripts just call them with the -h flag (python train_<algorithm>_cartpole.py -h).