This repository contains code implementing learning-based testing of deep RL agents playing Super Mario Bros., where we track metrics computed from neural networks used by the RL agents. It accompanies the paper "Bridging the Gap Between Models in RL: Test Models vs. Neural Networks", submitted to the AMOST workshop 2024.
The implementation is an adaptation of the previous work on Differential Safety Testing of Deep RL Agents. The deep RL code is based on the PyTorch tutorial on training deep RL agents for Super Mario Bros. and we use the gym-super-bros environment.
The dependencies required for setting up the experiments are given environment.yml, which can be used together with Conda. The source code in the root directory contains the main implementation files and the directory stats includes some scripts to analyze and plot experimental results.
This is an initial version of our implementation, setup, and experiments.