The dmc vision (task=dmc_walker_walk) has very bad performance #11
Comments
|
This looks like a serious issue. I will take a look. |
|
Hi @LYK-love , after a quick search I found that the most likely reason is the mismatch of SSM hyperparameters. For example, you are probably using the hidden size of 128 in each layer of SSM. This is not too small (we used 512 in the paper if I remember correctly). There can be other mismatches too. I will be doing some reproducibility checks in the next few days and then get back to you. You can ask me anything here in the meantime. |
|
Please note that the hidden size alone does not guarantee to fix everything, but you can try that in the meantime. |
|
I see the hidden_size attribute in the config file: rssm: {deter: 4096, units: 1024, hidden: 128, stoch: 32, classes: 32, ... }Here you set mmaze:
task: gym_memory_maze:MemoryMaze-9x9-v0
...
rssm.deter: 2048
rssm.units: 1024
.*\.cnn_depth: 48
.*\.mlp_units: 400
.*\.layers: 4
.*\.mlp_layers: 4
ssm.n_layers: 5
rssm.hidden: 512In this sense, hidden size=512 should reproduce the score for mmaze. Now I am trying to know if R2I can achieve original DreamerV3's performance with GRU. It should work since you didn't change the architecture except the backbone. My command is: current_date=$(date "+%Y%m%d-%H%M%S")
python recall2imagine/train.py \
--configs dmc_vision \
--ssm_type gru \
--wdb_name dmc_original_${current_date} \
--logdir ./logs/dmc_original_${current_date}Can you tell me what should I do in further to get the DreamerV3 score (with GRU backbone) on your code base? One thing worth consideration, as you mentioned, is the hidden size, since in DreamerV3, we have rssm: {deter: 4096, units: 1024, stoch: 32, classes: 32, ... }, and there is no hidden size attribute here. |
Whrn I use mimo as backbone and set hidden size=512, i.e., I use command: current_date=$(date "+%Y%m%d-%H%M%S")
python recall2imagine/train.py \
--configs dmc_vision \
--ssm_type mimo --rssm.hidden 512\
--wdb_name dmc_original_${current_date} \
--logdir ./logs/dmc_original_${current_date}I got After that, the program is totally stuck. Is this normal? |
|
Hey, I don't know if this is still a problem, but I think it is related to XLA/JAX and is also discussed in the original dreamerv3 repo. Maybe the solution works here as well. |
Hello, I ran R2I with command:
and got very low scores. According to DreamerV3 paper, it can achieve score > 900.
However, R2I can only achieve score < 200.
I think the hyperparameters are the same, in R2I's
config.yaml, I saw, this aligns with DreamerV3:
Can you explain why this happens? Maybe it's because the ssm backbone is not so good as gru in this task? Or there's sth wrong with my hyperparameters?
The text was updated successfully, but these errors were encountered: