Replication of People Construct Simplified Mental Representations to Plan

From Behind the Paper: People construct simplified mental representations to plan

Replication project of initial experiment from Ho, M. K., Abel, D., Correa, C. G., Littman, M. L., Cohen, J. D., & Griffiths, T. L. (2022). People construct simplified mental representations to plan. Nature, 606(7912), 129-136.

RPubs link to report: https://rpubs.com/justintheyang/wu2021replication
Link to original paper: Ho et al. - 2022 - People construct simplified mental representations.pdf

Project directory

• analyses: contains notebooks and files to analyse data
• experiments: contains configuration files and processed datasets
• experiment.psiturkapp: contains the experiment application code, used to host the experiment on Heroku
• vgc_project: a custom python package implementing value guided construal. Should be installed in the base directory in a virtual environment via pip install -e ./vgc_project
• original_paper: contains the original study
• writeup: contains the replication report

Experiment

Local server setup

You can run the study as a psiturk experiment by running:

$ cd experiment.psiturkapp
$ make dev

Doing so will start the server. The experiment can then be accessed locally here.

Heroku server setup

The Heroku setup consists of the following steps:

$ heroku create ho2022replication --buildpack heroku/python
$ heroku git:remote -a ho2022replication
$ heroku addons:create heroku-postgresql
$ git subtree push --prefix experiment.psiturkapp heroku main
$ heroku config:set ON_CLOUD=1

The Postgres database url can be found by querying:

heroku config

After the server is set up (check the server url with heroku domains), you may host the experiment on Prolific using the following URL parameters:

https://project_name.herokuapp.com/exp?CONFIG_FILE=exp1.0-config.json.zip&assignmentId={{%SESSION_ID%}}&hitId={{%STUDY_ID%}}&workerId={{%PROLIFIC_PID%}}&mode=live

Analyses

To process & get data, run the following sequence of commands in the project directory (ho2022/) using the jupyter_notebook conda environment:

on (jupyter_notebook) justyang:ho2022/ (main✗) $:

1. python project.py unzip_files
2. python experiments/exp1/pull_results.py pull_data_from_server --EXP_NAME=pilot_a --RAWRESULTS_DIR=data/pilot_a --CREDENTIALS_FILE=.creds.json
3. python experiments/exp1/pull_results.py parse_raw_datafiles --BASEGRIDS_FILENAME=experiments/mazes/mazes_0-11.json --RAWRESULTS_DIR=data/pilot_a --EXP_CONFIG_FILE=experiment.psiturkapp/static/config/exp1.0-config.json --EXP_RESULTS_DIR=experiments/exp1/data/pilot_a/
4. python experiments/exp1/prep_results.py do_exclusions --BASEGRIDS_FILENAME=experiments/mazes/mazes_0-11.json --RESULTSDIR=experiments/exp1/data/pilot_a/

Final participant sample size after exclusions:

(jupyter_notebook) justyang:ho2022/ (main✗) $ python experiments/exp1/prep_results.py do_exclusions --BASEGRIDS_FILENAME=experiments/mazes/mazes_0-11.json --RESULTSDIR=experiments/exp1/data/full_data_collection/                                                                                                                                                                                                                                      103 of 105 with >67.2 recorded trials
Performing Exclusions
96 of 103 passed 2 of 3 comprehension checks
non_consec_trials: 	0 of 1152
multiple_first_trials: 	0 of 1152
high_total_nav_time: 	20 of 1152
high_last_3_steps_rt: 	10 of 1152
high_initial_step_time: 	10 of 1152
high_non_initial_step_rt: 	40 of 1152
any_exclusion: 	59 of 1152
Participants with 67.2 of 84 valid response trials: 88 of 96
Total remaining participants: 88
Total remaining rounds: 1031
Total remaining responses: 7217