Data pipeline for Inagaki et al., 2018 & 2019 from Svoboda Lab.
This project presents a DataJoint pipeline design for the data accompanying the papers:
Hidehiko K. Inagaki, Lorenzo Fontolan, Sandro Romani & Karel Svoboda. "Discrete Attractor Dynamics Underlies Persistent Activity in the Frontal Cortex" (2019) Nature (https://doi.org/10.1038/s41586-019-0919-7)
Hidehiko K. Inagaki, Miho Inagaki, Sandro Romani and Karel Svoboda. "Low-Dimensional and Monotonic Preparatory Activity in Mouse Anterior Lateral Motor Cortex" (2018) Jneurosci (https://doi.org/10.1523/JNEUROSCI.3152-17.2018)
A DataJoint data pipeline has been constructed for this study, with the presented data ingested into this pipeline. This notebook demonstrates the queries, processing, and reproduction of several figures from the paper. From the pipeline, export capability to NWB 2.0 format is also available.
The dataset comprises of whole-cell recordings, extracellular recordings and spike sorted results of the mouse's ALM during a delay-response task. The behavior data includes detailed description of the trial structure (e.g. trial timing, trial instruction, trial response, etc.) and the timing of the lick events (e.g. lick left, lick right onset). The data also includes detailed information regarding the optogenetic perturbations on a per-trial basis.
The data in original MATLAB format (.mat) have been ingested into a DataJoint data pipeline presented below.
Data are also exported into NWB 2.0 format. See NWB export code here
This repository contains the Python 3.7 code of the DataJoint data pipeline design for this dataset, as well as scripts for data ingestions and visualization.
This repository contains the Python 3.7 code to convert the DataJoint pipeline into NWB 2.0 format (See https://neurodatawithoutborders.github.io/) Each NWB file represents one recording session. The conversion script can be found here
Data queries and usages are demonstrated in this Jupyter Notebook, where several figures from the paper are reproduced.
After cloning this repository, download the original data. Once downloaded, you should find 2 folders containing
intracellular and extracellular data, WholeCellData and SiliconProbeData respectively.
dj_local_conf.json is a configuration file for DataJoint, which minimally specifies the
database connection information, as well as several other optional configurations.
Create a new dj_local_conf.json at the root of your project directory (where you have this repository cloned),
with the following format:
{
"database.host": "database_hostname",
"database.user": "your_username_here",
"database.password": "your_password_here",
"database.port": 3306,
"database.reconnect": true,
"loglevel": "INFO",
"safemode": true,
"custom": {
"database.prefix": "inagaki2018_",
"extracellular_directory": ".../path_to_downloaded_data/SiliconProbeData",
"intracellular_directory": ".../path_to_downloaded_data/WholeCellData"
}
}Note: make sure to provide the correct database hostname, username and password. Then specify the path to the downloaded data directories for intracellular and extracellular data.
On a new terminal, navigate to the root of your project directory, then execute the following commands:
python scripts/ingest_extracellular.py
python scripts/ingest_wholecell.py
python scripts/populate.py
You now have a functional pipeline up and running, with data fully ingested. You can explore the data, starting with the provided demo notebook.
From your project root, launch jupyter notebook:
jupyter notebook
Data from this DataJoint pipeline can be exported in NWB 2.0 format using this datajoint_to_nwb.py script.
To perform this export for all ingested data, specify the export location (e.g. ./data/exported_nwb2.0), execute this command from the project root:
python scripts/datajoint_to_nwb.py ./data/exported_nwb2.0