Merge pull request #115 from ReproNim/nf-audio-codes

Research on how to best encode data within audio stream

tools/README.md

@@ -0,0 +1,168 @@
+# ReproStim Tools
+
+## Overview
+
+### A. Install Singularity
+
+#### On Linux (Ubuntu 24.04) :
+
+```shell
+wget -O- http://neuro.debian.net/lists/noble.de-m.libre | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list
+sudo apt-key adv --recv-keys --keyserver hkps://keyserver.ubuntu.com 0xA5D32F012649A5A9
+
+sudo apt-get update
+
+sudo apt-get install singularity-container
+```
+
+```shell
+$ singularity --version
+  singularity-ce version 4.1.1
+```
+
+### B. Install ReproNim Containers
+
+#### On Linux (Ubuntu 24.04) :
+
+Ensure DataLad is installed:
+
+```
+sudo apt-get install datalad
+```
+
+As next step install and download ReproNim containers:
+
+```
+datalad install https://datasets.datalad.org/repronim/containers
+datalad update
+cd ./containers/images/repronim
+datalad get .
+```
+
+Check that X11 system is used by default in Linux (Ubuntu 24.04), 
+psychopy will not work well with Wayland:
+
+```
+echo $XDG_SESSION_TYPE
+```
+
+It should return `x11`. If not, switch to X11:
+
+ - At the login screen, click on your username.
+ - Before entering your password, look for a gear icon or "Options" button at the bottom right corner of the screen.
+ - Click the gear icon, and a menu should appear where you can select either "Ubuntu on Xorg" or "Ubuntu on Wayland."
+ - Choose "Ubuntu on Xorg" to switch to X11.
+ - Enter your password and log in. You should now be running the X11 session.
+
+### C. Run ReproNim TimeSync Script
+
+Make sure the current directory is one under singularity container 
+path created in the previous step B:
+
+```shell
+cd ./containers/images/repronim
+```
+
+Run the script:
+
+```shell
+singularity exec ./repronim-psychopy--2024.1.4.sing ${REPROSTIM_PATH}/tools/reprostim-timesync-stimuli output.log 1
+``` 
+Where `REPROSTIM_PATH` is the local clone of https://github.com/ReproNim/reprostim repository.
+
+Last script parameter is the display ID, which is `1` in this case.  
+
+### D. Update Singularity Container Locally (Optionally)
+
+Optionally, you can update the container locally for development 
+and debugging purposes (with overlay):
+
+```shell
+singularity overlay create \
+  --size 1024 \
+  repronim-psychopy--2024.1.4.overlay
+
+sudo singularity exec \
+  --overlay repronim-psychopy--2024.1.4.overlay \
+  repronim-psychopy--2024.1.4.sing \
+  bash
+```
+As sample install some package:
+
+```shell
+apt-get update
+apt-get install pulseaudio-utils
+pactl
+exit
+```
+
+And now run the script with overlay:
+
+```shell
+singularity exec \
+  --cleanenv --contain \
+  -B /run/user/$(id -u)/pulse \
+  -B ${REPROSTIM_PATH} \
+  --env DISPLAY=$DISPLAY \
+  --env PULSE_SERVER=unix:/run/user/$(id -u)/pulse/native \
+  --overlay ./repronim-psychopy--2024.1.4.overlay \
+  ./repronim-psychopy--2024.1.4.sing \
+  ${REPROSTIM_PATH}/tools/reprostim-timesync-stimuli output.log 1
+``` 
+
+Where `/run/user/321/pulse` is sample external pulseaudio device path bound to the container. Usually 
+when you run the script w/o binding it will report error like:
+
+```shell
+Failed to create secure directory (/run/user/321/pulse): No such file or directory
+``` 
+
+NOTE: Make sure `PULSE_SERVER` is specified in the container environment and 
+points to the host pulseaudio server. e.g.:
+
+```shell
+export PULSE_SERVER=unix:/run/user/321/pulse/native
+```
+
+#### Dev Notes (for local PC altogether)
+
+```shell
+cd ~/Projects/Dartmouth/branches/datalad/containers/images/repronim
+export REPROSTIM_PATH=~/Projects/Dartmouth/branches/reprostim 
+
+singularity overlay create \
+  --size 1024 \
+  repronim-psychopy--2024.1.4.overlay
+
+sudo singularity exec \
+  --overlay repronim-psychopy--2024.1.4.overlay \
+  repronim-psychopy--2024.1.4.sing \
+  bash
+
+# execute in shell
+apt-get update
+apt-get install portaudio19-dev pulseaudio pavucontrol pulseaudio-utils
+pactl
+exit
+
+# make sure all python packages are installed
+sudo singularity exec \
+  --overlay repronim-psychopy--2024.1.4.overlay \
+  repronim-psychopy--2024.1.4.sing \
+  python3 -m pip install pyzbar opencv-python numpy click pydantic sounddevice scipy pydub pyaudio reedsolo psychopy-sounddevice 
+
+# and run the script
+rm output.log
+singularity exec \
+  --cleanenv --contain \
+  -B /run/user/$(id -u)/pulse \
+  -B ${REPROSTIM_PATH} \
+  --env DISPLAY=$DISPLAY \
+  --env PULSE_SERVER=unix:/run/user/$(id -u)/pulse/native \
+  --overlay ./repronim-psychopy--2024.1.4.overlay \
+  ./repronim-psychopy--2024.1.4.sing \
+  ${REPROSTIM_PATH}/tools/reprostim-timesync-stimuli output.log 1
+
+```
+
+

tools/audio-codes-notes.md

@@ -0,0 +1,198 @@
+# Audio Codes Notes
+
+## Installation
+
+```
+ python3.10 -m venv venv
+ source venv/bin/activate
+ pip install --upgrade pip
+ pip install -r audio-codes-requirements.txt
+```
+
+ On MacOS:
+```
+  brew install portaudio
+  pip install pyaudio
+```
+
+ On Linux:
+```
+  sudo apt-get install portaudio19-dev
+```
+
+## TODO:
+
+ Review `psychopy` and sound API and possibly use PTB's
+ facilities in PsychoPy for precise audio placement in time:
+   https://www.psychopy.org/download.html
+   https://psychopy.org/api/sound/playback.html
+
+ Look at watermark in audio.
+
+## PsychoPy, Sound, PTB
+
+### On MacOS:
+
+NOTE: PsychoPy (2024.2.3) current requirements limits/suggests
+      to Python version 3.10.
+
+Download and install the standalone package:
+PsychoPy 2024.2.3 modern (py3.10)
+https://github.com/psychopy/psychopy/releases/download/2024.2.3/StandalonePsychoPy-2024.2.3-macOS-py3.10.dmg
+
+Run PsychoPy and check the sound settings that `pbt` is
+set as `Audio Library`.
+
+Make sure Python 3.10 is installed and venv is explicitly created with it:
+```
+   python3.10 -m venv venv
+   source venv/bin/activate
+   pip install --upgrade pip
+   pip install -r audio-codes-requirements.txt
+```
+
+NOTE: first time PsychoPy is run, it may takes a long time to setup audio
+       download additional dependencies.
+
+### On Linux (Ubuntu 22.04):
+
+Create some folder for `psychopy` installation and init Python 3.10 venv:
+```
+   python3.10 -m venv venv
+   source venv/bin/activate
+```
+Then fetch a wxPython wheel for your platform from:
+https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ and
+download it locally to your machine.
+In my case it was `https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-22.04/wxPython-4.2.1-cp310-cp310-linux_x86_64.whl`
+downloaded to `wxPython/wxPython-4.2.1-cp310-cp310-linux_x86_64.whl` .
+
+Install sound dependencies (1st one may fail):
+```
+   sudo apt-get install libusb-1.0-0-dev portaudio19-dev libasound2-dev
+   pip install psychtoolbox
+```
+
+Install wxPython in `venv` with, e.g.:
+```
+   pip install wxPython/wxPython-4.2.1-cp310-cp310-linux_x86_64.whl
+```
+
+Then install `psychopy`:
+```
+   pip install psychopy
+```
+
+Run psychopy:
+```
+   psychopy
+```
+
+NOTE: PsychoPy PTB sound was non tested on Ubuntu 22.04, due to upgrade to 24.04.
+
+### On Linux (Ubuntu 24.04):
+
+Somehow was unable to run `python3.10` directly so installed it manually together with venv:
+
+```
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt update
+sudo apt install python3.10
+sudo apt install python3.10-venv
+```
+
+After this `python3.10` failed to create `venv` so used following commans to create it:
+
+```
+python3.10 -m venv --without-pip venv
+source venv/bin/activate
+curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+python get-pip.py
+```
+
+Then installed `wxPython` it can take up to 1 hour to compile it (there is no pre-built wheels for Ubuntu 24.04 ATM):
+
+```
+sudo apt update
+sudo apt install python3-dev python3-pip libgtk-3-dev
+pip install wxPython
+```
+
+Finally installed `psychopy`:
+
+```
+pip install psychopy
+```
+
+Applied security fix for audio scripts user, by adding current user to `audio` group:
+
+```
+sudo usermod -a -G audio $USER
+```
+
+and to adjust real-time permissions by creating a file `/etc/security/limits.d/99-realtime.conf` 
+
+```
+sudo vi /etc/security/limits.d/99-realtime.conf
+```
+
+with the following content:
+
+```
+@audio   -  rtprio     99
+@audio   -  memlock    unlimited
+```
+
+Then rebooted the system.
+
+NOTE: PsychoPy PTB still doesn't work on Ubuntu 24.04 and script produces error, TBD:
+
+```
+[DEBUG] play sound with psychopy ptb
+Failure: No such entity
+Failure: No such entity
+```
+
+## NeuroDebian/Singularity
+
+On Linux (Ubuntu 22.04) :
+
+Was unable to install `singularity-container` from `neurodebian` repository after multiple attempts, so finally upgraded Linux 22.04 to 24.04.
+
+
+On Linux (Ubuntu 24.04) :
+
+```
+wget -O- http://neuro.debian.net/lists/noble.de-m.libre | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list
+sudo apt-key adv --recv-keys --keyserver hkps://keyserver.ubuntu.com 0xA5D32F012649A5A9
+
+sudo apt-get update
+
+sudo apt-get install singularity-container
+```
+
+```
+singularity --version
+  singularity-ce version 4.1.1
+```
+
+## Summary
+  - `PyDub` allows you to generate simple tones easily.
+  -  FSK modulation can be achieved using `numpy` and
+  `scipy` to create varying frequency tones based on
+  binary input.
+  - Optionally `DTMF` encoding is implemented using
+  predefined frequency pairs, and you can detect
+  DTMF tones by analyzing the audio input.
+
+  - Chirp SDK:
+    Chirp.io / https://www.sonos.com/en/home
+    https://github.com/chirp
+
+  - GNU Radio - can be used to encode/modulate/demodulate.
+    https://www.gnuradio.org/
+    Supports Frequency Shift Keying (FSK),
+    Phase Shift Keying (PSK), or Amplitude Modulation (AM).
+
+  - `reedsolo` can be used for ECC (Error Correction Codes).
+

tools/audio-codes-requirements.txt

@@ -0,0 +1,14 @@
+# draft requirements for audio-codes.py
+pyzbar>=0.1.9
+opencv-python>=4.9.0.80
+numpy>=1.26.4
+click>=8.1.7
+pydantic>=2.7.1
+sounddevice>=0.5.1
+scipy>=1.14.1
+pydub>=0.25.1
+pyaudio>=0.2.14
+reedsolo>=1.7.0
+psychopy
+psychopy-sounddevice
+qrcode