Our code is a modified version of the original code for the control tasks paper.
Warning! We used code from the codalab worksheets and NOT from the repo, because we used the code referred to in the paper. We do not know if the code is different from the one in the repo.
Download data and ELMo vectors using our link or from the original codalab worksheets: the data is the same.
Folder ./mdl_configs contains examples of config files for variational and online codings.
You have to change the lines marked by the # CHANGE THIS: ... comment:
-
path to the ptb data,
-
path to the elmo vectors,
-
layer number,
-
output directory with checkpoints and logs.
If you want, you can also change the lines marked by the # CHANGE THIS (IF YOU REALLY WANT) comment:
-
number of probe layers
-
size of the probe hidden layer.
How to run an experiment:
python /path_to_this_repo/control_tasks/run_experiment.py /path_to_this_repo/mdl_configs/bayes_l0.yml
Folder ./mdl_eval_notebooks contains notebooks shoing how to evaluate online and variational codelength from the logs, and how to get pruned probe architecture for the variational probe.