ScanImage dataset stack_type attribute not parsed from yaml

[cypranowska]

Setting 'stack_type'='zstack' in the yaml is causing issues on ScanImage datasets with multiple files on the same acquisition. Triggers warning on line 174 of flexiznam/schema/scanimage_data.py.

[ablot]

@cypranowska Do you have an example yaml file that I could parse to debug?

[cypranowska]

I think the only yaml I have for a z-stack split into multiple tiffs is at /camp/home/users/cypranc/mouse_yamls/s20230605_valid.yml

[ablot]

I cannot reproduce this behaviour for now. If you can provide a minimal code example that would be great

[cypranowska]

Sorry for leaving this issue hanging for so long D: Calling flexiznam yaml-to-flexilims -s test.yml -r /nemo/lab/znamenskiyp/data/instruments/raw_data/BRAC9331.4a/S20240510/ -c skip ...

with the contents of test.yml

attributes: null
datasets:
  ref_tile_00001:
    attributes:
      acq_num: '00001'
      acq_uid: ref-tile_00001
      csv_files: {}
      file_stem: ref-tile
      stack_type: zstack
      tif_files:
      - ref-tile_00001_00019.tif
      - ref-tile_00001_00008.tif
      - ref-tile_00001_00014.tif
      - ref-tile_00001_00001.tif
      - ref-tile_00001_00007.tif
      - ref-tile_00001_00020.tif
      - ref-tile_00001_00006.tif
      - ref-tile_00001_00016.tif
      - ref-tile_00001_00010.tif
      - ref-tile_00001_00002.tif
      - ref-tile_00001_00012.tif
      - ref-tile_00001_00017.tif
      - ref-tile_00001_00004.tif
      - ref-tile_00001_00003.tif
      - ref-tile_00001_00018.tif
      - ref-tile_00001_00009.tif
      - ref-tile_00001_00005.tif
      - ref-tile_00001_00011.tif
      - ref-tile_00001_00015.tif
      - ref-tile_00001_00013.tif
    created: '2024-05-10 13:29:15'
    dataset_type: scanimage
    genealogy:
    - S20240510
    - ref-tile_00001
    is_raw: 'yes'
    origin_id: null
    path: /nemo/lab/znamenskiyp/data/instruments/raw_data/projects/depth_mismatch_seq/BRAC9331.4a/S20240510
genealogy: null
mouse: BRAC9331.4a
notes: null
path: /nemo/lab/znamenskiyp/data/instruments/raw_data/projects/depth_mismatch_seq/BRAC9331.4a/S20240510
project: depth_mismatch_seq
recordings: {}
samples: {}
session: S20240510

... will produce the error message

/camp/home/cypranc/.conda/envs/iss-preprocess/lib/python3.10/site-packages/flexiznam/schema/scanimage_data.py:111: UserWarning: No `stack_type` provided for SI dataset ('S20240510', 'ref-tile_00001'). Set to default: calcium
  output[acq_id] = ScanimageData(
Warning: key `ref-tile_00001` contains special characters and is unvalid JSON. Will use ref_tile_00001 instead
Warning: csv_files is an empty structure and will be uploaded as `None`

In the interim, I will continue fixing this with update_flexilims but I'd avoid trying to reproduce the error until 13/05 because I'm actually trying to register this dataset.