A question on the dim and dtype of action.

[lk1983823]

I have successfully run the cartpole_simulink.ipynb file. Now I have two questions:

1. In the step method in cartpole_simulink.py, I find you define action = int(action) in line 119. Is it due to just the action.space requirentment or the TCP/IP sending requirements. If I train a RL model with float type action, will it have any problem?
2. In the cartpole example, the dimension of action is only one. If I have action with multiple dimensions, will there be any fix in simulink_block_lib? Can you help to specify how to fix it? Or suggest to add an example with more than one dimension with float dtype.
   Thanks.

[johbrust]

Regarding question 1: Yes, action = int(action) is specific to the environment and its action space. Floats are no problem.

Regarding question 2: You can define an input vector (array with one dimension) with multiple actions. But you have to set this also in your Simulink model in the TCP/IP Input block (see description here for block parameter Number of input signals).

I created an example here in a separate branch with 2 float actions for you to check out.

[lk1983823]

Regarding question 1: Yes, action = int(action) is specific to the environment and its action space. Floats are no problem.

Regarding question 2: You can define an input vector (array with one dimension) with multiple actions. But you have to set this also in your Simulink model in the TCP/IP Input block (see description here for block parameter Number of input signals).

I created an example here in a separate branch with 2 float actions for you to check out.

Thanks ! The toy example is very helpful !

[lk1983823]

I add step_size in this simple_reacher_simulink in the attachment and run it successfully.
simple_reacher_simulink_with_step_size.tar.gz

[johbrust]

The difference is that Simulink.Parameter allows for more detailed specifications, as you can see below. You can specify minimum and maximum values or a unit and give a description of the parameter.

You can create new a Simulink.Parameter from the block parameter dialogue by selecting Simulink.Parameter for the Value in the Create dialogue:

You can also create parameters in the model explorer through Add → Simulink Parameter and afterwards select it in the block dialogue:

[lk1983823]

The difference is that Simulink.Parameter allows for more detailed specifications, as you can see below. You can specify minimum and maximum values or a unit and give a description of the parameter.

You can create new a Simulink.Parameter from the block parameter dialogue by selecting Simulink.Parameter for the Value in the Create dialogue:

You can also create parameters in the model explorer through Add → Simulink Parameter and afterwards select it in the block dialogue:

Thanks! I have updated the attachment.