error in prediction sample code

[bhralzz]

Hi
Dear Dominik
The network test code has been tested as below just on 3 sample case from kits:

**from miscnn import *

Create a Data I/O interface for kidney tumor CT scans in NIfTI format

from miscnn.data_loading.interfaces import NIFTI_interface
interface = NIFTI_interface(pattern="case_000[0-9]*", channels=1, classes=3)

Initialize data path and create the Data I/O instance

data_path = "dataset_temp/"
data_io = Data_IO(interface, data_path)

Create a Preprocessor instance to configure how to preprocess the data into batches

pp = Preprocessor(data_io, batch_size=4, analysis="patchwise-crop",
patch_shape=(80,160,160))

Create a deep learning neural network model with a standard U-Net architecture

from miscnn.neural_network.architecture.unet.standard import Architecture
unet_standard = Architecture()
model = Neural_Network(preprocessor=pp, architecture=unet_standard)

Training the model with 3samples for 5epochs

sample_list = data_io.get_indiceslist()
model.train(sample_list[0:2], epochs=5)

Predict the segmentation for samples

pred = model.predict(sample_list[2], return_output=True)

but the final output presented as below:

1/1 [==============================] - 40s 40s/step - loss: 2.4510 - dice_soft: 0.1830
Epoch 2/5
1/1 [==============================] - 23s 23s/step - loss: 2.4692 - dice_soft: 0.1769
Epoch 3/5
1/1 [==============================] - 28s 28s/step - loss: 2.4522 - dice_soft: 0.1826
Epoch 4/5
1/1 [==============================] - 19s 19s/step - loss: 2.4147 - dice_soft: 0.1951
Epoch 5/5
1/1 [==============================] - 27s 27s/step - loss: 2.4354 - dice_soft: 0.1882**
.
.
.
.
ValueError: Image could not be found "dataset_temp/c"

could you please check this?

[muellerdo]

Hey @bhralzz,

mhm looks like you have some problems in the data loading process.

Can you verify that you downloaded the KITS19 dataset correctly?
The data should look like this as explained in the MIScnn wiki:

data/
     sample001/imaging.nii.gz
               segmentation.nii.gz
     sample002/imaging.nii.gz
               segmentation.nii.gz
     sample003/imaging.nii.gz
               segmentation.nii.gz
     ...

I would also refer to our KiTS19 example JupyterNotebook: https://github.com/frankkramer-lab/MIScnn/blob/master/examples/KiTS19.ipynb

If running

sample_list = data_io.get_indiceslist()
sample_list.sort()
print("All samples: " + str(sample_list))

on the Data IO object, you should obtain the indices of all samples. If not, the dataset was not correctly downloaded or the path to the directory is incorrect.

Cheers,
Dominik

[bhralzz]

Dear Dominik,

Yes, all data samples downloaded correctly.
I guess that the error occured in acumulating the predicted batches into actual segmentation.
could you please check there in postprocessing?
otherwise, I have an idea,
If possible please upload a zipped file, tested code + 3 sample data (2 for training + 1 for testing).
Just to check the mechanism in small size dataset!
Thanks a lot,
Behrouz

[muellerdo]

Hey @bhralzz,

please attach your code and complete error log.

The KiTS19 Jupyter Notebook is a fully functional example for the complete KiTS19 dataset.

The best solution would be if you attach an Google Colab Notebook with this error for reproducing this bug. From the ValueException you posted I strongly suspect that something is incorrect in the data folder or during data loading. Batches & predictions are generated in separated folders.

Having a look on your complete code & error message will get things clearer. I'm optimistic that we are get your kits19 MIScnn instance running!

Cheers,
Dominik

[bhralzz]

simply just put this code beside miscnn source code :

from miscnn import *
from miscnn.data_loading.interfaces import NIFTI_interface
interface = NIFTI_interface(pattern="case_000[0-9]*", channels=1, classes=3)
data_path = "small_dataset/"
data_io = Data_IO(interface, data_path)

pp = Preprocessor(data_io, batch_size=4, analysis="patchwise-crop",
patch_shape=(80,160,160))

from miscnn.neural_network.architecture.unet.standard import Architecture
unet_standard = Architecture()
model = Neural_Network(preprocessor=pp, architecture=unet_standard)

sample_list = data_io.get_indiceslist()
model.train(sample_list[0:2], epochs=3)

pred = model.predict(sample_list[0], return_output=True) # error produced in this mode but below without error
#pred = model.predict(sample_list[0:2], return_output=True)

should we pass sample_list in list mode even single sample?

[muellerdo]

Hey @bhralzz,

the samples should be passed as a list.
Passing single samples should, thus, look liked this model.predict(["my_single_sample"])

Concerning the error message: Did you already fixed it?

Cheers,
Dominik