Chapter 5

[MFKruger]

I am using the gymnasium library and the "FrozenLake-v1" environment.
I had to make some minor changes, like
new_state, reward, is_done, _, _ = self.env.step(action)

When running the "01_frozenlake_v_iteration" code in a Jupyter notebook I get the following error message:
TypeError Traceback (most recent call last)
Cell In[1], line 81
79 while True:
80 iter_no += 1
---> 81 agent.play_n_random_steps(100)
82 agent.value_iteration()
84 reward = 0.0

Cell In[1], line 25
23 action = self.env.action_space.sample()
24 new_state, reward, is_done, _, _ = self.env.step(action)
---> 25 self.rewards[(self.state, action, new_state)] = reward
26 self.transits[(self.state, action)][new_state] += 1
27 self.state = self.env.reset()
28 if is_done else new_state

TypeError: unhashable type: 'dict'

Any help will be much appreciated!

[Simon-dl]

If I remember correctly its because when you reset the environment in the new version of gym it returns two values, {obs, probs}. so the code will work fine on first reset but when the episode ends and the environment resets again it doesn't know what to do with the extra probs value. Try using the python debugger and see if that's what it is. if it is just get obs alone from the reset with self.env.reset()[0]

[MFKruger]

Simon: Thank you!