🐛 [slice timing] unmatched tpattern and slices

[suxpert]

Describe the bug

I have some functional data which covers only half of the brain, and with slice setting to almost perpendicular to anterior-posterior, i.e., coronal slices.

fmriprep works fine for this data, and because I failed run C-PAC on top of fmriprep's result, I tried with the raw data in BIDS. Still without any lucky, C-PAC failed with following errors:

Traceback (most recent call last):
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/plugins/linear.py", line 47, in run
    node.run(updatehash=updatehash)
  File "/code/CPAC/pipeline/nipype_pipeline_engine/engine.py", line 443, in run
    return super().run(updatehash)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node slice_timing.

Cmdline:
        3dTshift -prefix sub-0358_task-rest_run-01_bold_resample_calc_tshift.nii.gz -tpattern @/out/working/pipeline_cpac_preproc/cpac_sub-0358_ses-1/_scan_rest_run-01/bold_scan_params_sub-0358_ses-1/tpattern.txt -TR 1.5s /out/working/pipeline_cpac_preproc/cpac_sub-0358_ses-1/func_slice_timing_correction_122/_scan_rest_run-01/slice_timing/sub-0358_task-rest_run-01_bold_resample_calc.nii.gz
Stdout:

Stderr:
        ++ 3dTshift: AFNI version=AFNI_23.3.09 (Dec  8 2023) [64-bit]
        ** FATAL ERROR: tpattern file /out/working/pipeline_cpac_preproc/cpac_sub-0358_ses-1/_scan_rest_run-01/bold_scan_params_sub-0358_ses-1/tpattern.txt has 66 values but have 96 slices
        ** Program compile date = Dec  8 2023
Traceback:
        Traceback (most recent call last):
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
            setattr(outputs, key, val)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
            value = super(File, self).validate(objekt, name, value, return_pathlike=True)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
            self.error(objekt, name, str(value))
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
            raise TraitError(
        traits.trait_errors.TraitError: The 'out_file' trait of a TShiftOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/out/working/pipeline_cpac_preproc/cpac_sub-0358_ses-1/func_slice_timing_correction_122/_scan_rest_run-01/slice_timing/sub-0358_task-rest_run-01_bold_resample_calc_tshift.nii.gz' <class 'str'> was specified.

        During handling of the above exception, another exception occurred:

        Traceback (most recent call last):
          File "/code/CPAC/pipeline/nipype_pipeline_engine/monkeypatch.py", line 43, in run
            outputs = self.aggregate_outputs(runtime)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
            raise FileNotFoundError(msg)
        FileNotFoundError: No such file or directory '/out/working/pipeline_cpac_preproc/cpac_sub-0358_ses-1/func_slice_timing_correction_122/_scan_rest_run-01/slice_timing/sub-0358_task-rest_run-01_bold_resample_calc_tshift.nii.gz' for output 'out_file' of a TShift interface


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/code/CPAC/pipeline/cpac_pipeline.py", line 578, in run_workflow
    workflow_result = workflow.run(plugin=plugin,
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/plugins/linear.py", line 82, in run
    raise error from cause
RuntimeError: 2 raised. Re-raising first.

#1700 might be the same issue, but 2 years have passed without further update.

To reproduce

No response

Preconfig

• default
• abcd-options
• anat-only
• blank
• ccs-options
• fmriprep-options
• fx-options
• monkey
• monkey-ABCD
• ndmg
• nhp-macaque
• preproc
• rbc-options
• rodent

Custom pipeline configuration

No response

Run command

docker run -it --rm -u $(id -u) -v $BIDS:/data:ro -v $BIDS/derivatives/cpac:/out fcpindi/c-pac --preconfig preproc /data /out participant

Expected behavior

I would expect the preproc pipeline be able to run successfully since this is very simple BIDS dataset that has been validated via bida-validator and is nothing wrong for fmriprep.

Acceptance criteria

• C-PAC correctly applied slice timing correction and go without further issue;
• C-PAC accept turning off slice timing correction from command line rather than edit the terrified pipeline config files (which is hard for me because it is too complex for beginners and without detailed documents);
• C-PAC was easily configurable to run with fmriprep's output from command line, without issues like 🐛 [fmriprep-ingress] pipeline failed to start #2144

Screenshots

Here are crash and log files:

• failedToStart.log
• pypeline.log
• callback.log
• crash-20240829-162213-c-pac_user-slice_timing.a1-bca24234-d272-43bb-9437-7d526e011088.txt
• cpac_data_config_f2f113b2_2024-08-29T14-06-33Z.yml
• cpac_pipeline_config_f2f113b2_2024-08-29T14-06-33Z.yml
• cpac_pipeline_config_f2f113b2_2024-08-29T14-06-33Z_min.yml

C-PAC version

v1.8.7

Container platform

Docker

Docker and/or Singularity version(s)

Docker version 26.1.5-ce

Additional context

$ fslsize bids-new3/sub-0151/func/sub-0151_task-rest_run-01_bold
dim1            116
dim2            96
dim3            66
dim4            200
pixdim1         1.458330
pixdim2         1.458330
pixdim3         1.500000
pixdim4         1.500000

$ jq -r '.SliceTiming | length' bids-new3/sub-0151/func/sub-0151_task-rest_run-01_bold.json
66

[tamsinrogers]

Hi @suxpert, thanks for reaching out.

C-PAC correctly applied slice timing correction and go without further issue

It looks like your data data has the slice-timing info in the sidecar, but it’s either missing from the header or on a different axis than expected. Since fmriprep gets the slicetiming from the sidecar and you already have it in the sidecar, you can

1. extract that array into a tpattern file for AFNI 3dTShift (just numbers and spaces in a text file)
2. bind that tpattern file to your container
3. run with a config file like:

FROM: preproc
functional_preproc:
    slice_timing_correction:
        tpattern: path/to/that/tpattern/file.txt

C-PAC accept turning off slice timing correction from command line rather than edit the terrified pipeline config files (which is hard for me because it is too complex for beginners and without detailed documents)

A pipeline config like this should run the preproc preconfig minus slice timing correction:

FROM: preproc
functional_preproc:
    slice_timing_correction:
        run: Off

There is documentation here for making small changes to a configuration and here for slice-timing settings. Looking at the slice-timing docs, it looks like instead of putting the tpattern file in the pipeline config, you could make that file and put its path in the “Acquisition” column of your scan_parameters.csv or scan_parameters_multiscan.csv file.
Note: Scan parameter information specified for slice timing correction will override the settings specified in the pipeline configuration YAML.

Hopefully some of this helps with the issue you are having. It would also be helpful for us to know know what your expectations are for the CLI and documentation here, specifically. Any additional information helps us make improvements to C-PAC!

[suxpert]

• I was able to run C-PAC with slice-timing correction turned off (as a workaround).
• It does not help if I extract the slice timing information and binding it to docker, since the error of 3dTshift is number mismatch, where I have only 66 slices but it requires 96, i.e, on a different axis than expected.

** FATAL ERROR: tpattern file /..../tpattern.txt has 66 values but have 96 slices

I regard the config file as terrified since it:

• was not very-well structured like toml et al., since yaml use indent for hierarchy, but too many levels;
• was too complex since there are too many things to write, but I did not find the logic of all those stuffs;
• without a comprehensive documents telling that:
  • How many top-level keys, what are they, and what are they used for;
  • How to combine such top-level keys to realize some function;
  • What options/sub-level functions included in one certain top-level keys, what do they mean, and to choose what values (and why)
• Since the configure files can be very deep (nested according to the document), such a tree (or forest) makes me easily lost.
• For the Web-based Graphics UI, I entered here, clicked at the duplicate, then view/edit, then the only things that I can do is to rename the duplicated config, or modify the from tab by entering new text, all other tabs are empty.

[tamsinrogers]

Hi @suxpert,

Thank you for the update. If specifying the tpattern in the following ways has not resolved this issue, we can replicate on our end and get back to you with some more information. Specifying your intended use case, if you are able, will also help us do this!

1. in config file

FROM: preproc
functional_preproc:
    slice_timing_correction:
        tpattern: path/to/that/tpattern/file.txt 

2. make that file and put its path in the “Acquisition” column of your scan_parameters.csv or scan_parameters_multiscan.csv file

Here is some additional information relating to C-PAC configuration:

• C-PAC makes it possible to use a single configuration file to launch a factorial number of pipelines differing with respect to specific processing steps (e.g., spatial/temporal filter settings, global correction strategies, motion correction strategies, group analysis models)
• descriptions of preconfigs
• Documentation for configuring pre-and post-processing and derivatives,
• If you want to base a pipeline on another pipeline configuration YAML file, you can specify FROM: /path/to/pipeline.yml in your pipeline configuration file.
• You can use the name of a preconfigured pipeline instead of a filepath if you want to base a configuration file on a preconfigured pipeline. If FROM is not specified, the pipeline will be based on the blank pipeline.
• C-PAC will include all expected keys from the pipeline file specified in FROM (or the blank pipeline if none is specified).” Any keys specified in a pipeline configuration file will take precedence over the same key in the FROM base configuration, but all omitted keys will retain their values from the FROM base configuration.
• From terminal, you can quickly generate a default pipeline configuration YAML file template in the directory you are in:
  cpac utils pipe-config new-template. You can then edit the file as needed. For values that you want to leave at the default, you can either leave the key as-is, or you can remove the key, and C-PAC will automatically use value from the default pipeline configuration (or from the pipeline specified in FROM).
• There is additional comprehensive documentation in the comments for the default pipeline. Specifically, how many keys there are at all levels, what they are, and what they’re used for.