Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update instructions for creating a new dataset #107

Merged
merged 5 commits into from
Sep 20, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
data/
orgdata/
orgdata
output/
tmp*/
err*.txt
@@ -13,3 +14,6 @@ __pycache__/
*.ahv
*.hv
*.v
*.csv
# Tensorboard files
events.*
35 changes: 35 additions & 0 deletions SIRF_data_preparation/README.md
Original file line number Diff line number Diff line change
@@ -25,3 +25,38 @@ for the Siemens mMR NEMA IQ data (on Zenodo):
- `download_Siemens_mMR_NEMA_IQ.py`: download and extract
- `prepare_mMR_NEMA_IQ_data.py`: prepare the data (prompts etc)
- `BSREM_*.py`: functions with specific settings for a particular data-set

# Steps to follow to prepare data
If starting from Siemens list-mode data and letting SIRF take care of scatter etc, check for instance [Siemens_mMR_ACR/README.md](steps for Siemens mMR ACR). If pre-prepared data are given, check that naming of all files is correct. KT normally puts all data
in `~/devel/PETRIC/data/<datasetname>` with `datasetname` following convention of `scanner_phantom-name` as others (instructions below and indeed some scripts might assume this location). Change working directory to where data sits and add PETRIC to your python-path, e.g.
```
PYTHONPATH=~/devel/PETRIC:$PYTHONPATH`
```

1. Run initial [data_QC.py](data_QC)
```
python -m SIRF_data_preparation.data_QC
```

2. Run [create_initial_images.py](create_initial_images).
```
python -m SIRF_data_preparation.create_initial_images --template_image=<some_image>
```
where the template image is one of the given VOIs (it does not matter which one, as they should all have the same geometry). (If you need to create VOIs yourself, you can use `None` or the vendor image).
3. Edit `OSEM_image.hv` to add modality, radionuclide and duration info which got lost (copy from `prompts.hs`)
4. Edit [dataset_settings.py](dataset_settings.py) for subsets (used by our reference reconstructions only, not by participants).
5. Edit [../petric.py](petric.py) for slices to use for creating figures (`DATA_SLICES`). Note that `data_QC` outputs centre-of-mass of the VOIs, which can be helpful for this.
6. Run [data_QC](data_QC) which should now make more plots. Check VOI alignment etc.
```
python -m SIRF_data_preparation.data_QC --dataset=<datasetname>
```
7. `cd ../..`
8. `python -m SIRF_data_preparation.run_OSEM <datasetname>`
9. `python -m SIRF_data_preparation.run_BSREM <datasetname>`
10. Adapt [plot_BSREM_metrics.py](plot_BSREM_metrics.py) (probably only the `<datasetname>`) and run interactively.
11. Copy the BSREM ` iter_final` to `data/<datasetname>/PETRIC/reference_image`, e.g.
```
stir_math data/<datasetname>/PETRIC/reference_image.hv output/<datasetname>/iter_final.hv
```
12. `rm output/<datasetname>/*ahv`, check its `README.md` etc
13. Transfer to web-server
8 changes: 4 additions & 4 deletions SIRF_data_preparation/Siemens_mMR_ACR/README.md
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ ugly and temporary
Steps to follow:
1. `python SIRF_data_preparation/Siemens_mMR_ACR/download.py`
2. `python SIRF_data_preparation/Siemens_mMR_ACR/prepare.py` (As there is no useful mumap, this will **not** do attenuation and scatter estimation)
3. `python SIRF_data_preparation/create_initial_images.py data/Siemens_mMR_ACR/final --template_image=None` (Reconstruct at full FOV size)
3. `python SIRF_data_preparation.create_initial_images data/Siemens_mMR_ACR/final --template_image=None` (Reconstruct at full FOV size)
4. `mv data/Siemens_mMR_ACR/final/OSEM_image.* data/Siemens_mMR_ACR/processing` (this is really an NAC image and ideally would be renamed)
5. `python SIRF_data_preparation/Siemens_mMR_ACR/register_mumap.py` (output is orgdata/Siemens_mMR_ACR/processing/reg_mumap.hv etc)
6. However, registration failed, so this needs a manual intervention step:
@@ -24,9 +24,9 @@ Steps to follow:
stir_math --accumulate orgdata/Siemens_mMR_ACR/processing/reg_mumap.hv orgdata/Siemens_mMR_ACR/output/ACR_hardware-to-STIR.nii.gz
```
7. 11. `python SIRF_data_preparation/run_OSEM.py Siemens_mMR_ACR`--end 200` (As there is now a useful mumap, this will now do attenuation and scatter estimation)
8. `python SIRF_data_preparation/create_initial_images.py data/Siemens_mMR_ACR --template_image=../../orgdata/Siemens_mMR_ACR/output/sampling_masks/acr-all-sampling-0-2mm_dipy.nii`
9. `python SIRF_data_preparation/data_QC.py --srcdir='data/Siemens_mMR_ACR' --transverse_slice=99`
10. edit `SIRF_data_prepatation/dataset_settings.py` for subsets etc. edit `OSEM_image.hv` to add modality, radionuclide and duration info which got lost.
8. `python -m SIRF_data_preparation.create_initial_images data/Siemens_mMR_ACR --template_image=../../orgdata/Siemens_mMR_ACR/output/sampling_masks/acr-all-sampling-0-2mm_dipy.nii`
9. `python -m SIRF_data_preparation.data_QC --srcdir='data/Siemens_mMR_ACR' --transverse_slice=99`
10. edit `petric.py` and `SIRF_data_preparation/dataset_settings.py` for subsets etc. edit `OSEM_image.hv` to add modality, radionuclide and duration info which got lost.
11. `python SIRF_data_preparation/Siemens_mMR_ACR/VOI_prep.py`
12. `python SIRF_data_preparation/run_OSEM.py Siemens_mMR_ACR`
13. `python SIRF_data_preparation/run_BSREM.py Siemens_mMR_ACR`
2 changes: 1 addition & 1 deletion SIRF_data_preparation/Siemens_mMR_ACR/prepare.py
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
import logging
import os

from SIRF_data_preparation.data_utilities import prepare_challenge_Siemens_data, the_data_path, the_orgdata_path
from ..data_utilities import prepare_challenge_Siemens_data, the_data_path, the_orgdata_path

scanID = 'Siemens_mMR_ACR'

Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
import logging
import os

from .data_utilities import prepare_challenge_Siemens_data, the_data_path
from ..data_utilities import prepare_challenge_Siemens_data, the_data_path, the_orgdata_path

this_directory = os.path.dirname(__file__)
repo_directory = os.path.dirname(this_directory)
4 changes: 2 additions & 2 deletions SIRF_data_preparation/create_initial_images.py
Original file line number Diff line number Diff line change
@@ -8,15 +8,15 @@

Options:
-t <template_image>, --template_image=<template_image> filename (relative to <data_path>) of image to use
for data sizes [default: PETRIC/VOI_whole_phantom.hv]
for data sizes [default: PETRIC/VOI_whole_object.hv]
-s <xy-size>, --xy-size=<xy-size> force xy-size (do not use when using VOIs as init) [default: -1]
-S <subsets>, --subsets=<subsets> number of subsets [default: 2]
-i <subiterations>, --subiterations=<subiterations> number of sub-iterations [default: 14]
"""
# Copyright 2024 Rutherford Appleton Laboratory STFC
# Copyright 2024 University College London
# Licence: Apache-2.0
__version__ = '0.1.0'
__version__ = '0.2.0'

import logging
import math
11 changes: 8 additions & 3 deletions SIRF_data_preparation/data_QC.py
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@

Options:
-h, --help
--srcdir=<path> pathname (must have trailing slash!) [default: ./]
--srcdir=<path> pathname. Will default to current directory unless dataset is set
--skip_sino_profiles do not plot the sinogram profiles
--transverse_slice=<i> idx [default: -1]
--coronal_slice=<c> idx [default: -1]
@@ -17,7 +17,7 @@
"""
# Copyright 2024 University College London
# Licence: Apache-2.0
__version__ = '0.3.0'
__version__ = '0.4.0'

import os
import os.path
@@ -31,6 +31,7 @@

import sirf.STIR as STIR
from SIRF_data_preparation.dataset_settings import get_settings
from SIRF_data_preparation.data_utilities import the_data_path

STIR.AcquisitionData.set_storage_scheme('memory')

@@ -163,11 +164,15 @@ def main(argv=None):
slices["sagittal_slice"] = literal_eval(args['--sagittal_slice'])

if (dataset):
if srcdir is None:
srcdir = the_data_path(dataset)
settings = get_settings(dataset)
for key in slices.keys():
if slices[key] == -1 and key in settings.slices:
slices[key] = settings.slices[key]
print(slices)
else:
if srcdir is None:
srcdir = os.getcwd()

if not skip_sino_profiles:
acquired_data = STIR.AcquisitionData(os.path.join(srcdir, 'prompts.hs'))