Quick question re: final Dice coefficient values

[lcaronson]

Hello Dominik,

In your paper in BMC you mention kidney Dice coefficients and tumor Dice coefficients. How did you get the algorithm to output a separate Dice coefficient for kidneys and tumors?

Thanks for all of your help and hard work!

Luke

[muellerdo]

Hey @lcaronson,

How I do it

How did you get the algorithm to output a separate Dice coefficient for kidneys and tumors?

I don't. I compute them manually after the prediction.

So my procedure is something like that:

• Run model.predict and storing predictions on disk
• Iterate over all predictions/samples with a for loop and
• Loading ground truth & prediction
• Compute metrics like dice between these two
• Append score to result_list
• After the loop, compute mean on result_list or plot it like a box plot

Example code from a student of mine in the RetinalSeg example:

# Make prediction for test set
model.predict(test)

# Define Calculation of dice similarity coefficient
def calc_DSC(truth, prediction, classes):
    dice_scores = []
    # Iterate over each class
    for i in range(classes):
      try:
          gt = np.equal(truth, i)
          pd = np.equal(prediction, i)
          # Calculate Dice
          dice = 2*np.logical_and(pd, gt).sum() / (pd.sum() + gt.sum())
          dice_scores.append(dice)
      except ZeroDivisionError:
          dice_scores.append(0.0)
    # Return computed dice similarity coefficients
    return dice_scores

dices_vessel = []
dices_background = []

# Compute dice similarity coefficient for all test images
for i in range(len(test)):
  sample = data_io.sample_loader(test[i], load_seg=True, load_pred=True)
  dice_scores = calc_DSC(sample.seg_data, sample.pred_data, classes=2)
  dices_vessel.append(dice_scores[1])
  dices_background.append(dice_scores[0])

# Print dice score mean
import numpy as np
print("Segmented vessels:", np.mean(dices_vessel))
print("Segmented background:",  np.mean(dices_background))

# Create boxplots
fig, ax = plt.subplots()
ax.boxplot([dices_vessel, dices_background])

# Set title and labels
ax.set_title('Evaluation plot for vessel segmentation')
ax.set_ylabel('Dice similarity coefficient')
ax.set_xticklabels(["Vessels", "Background"])

# Set range of y axis from 0 to 1
plt.ylim(0, 1)

# Show Box plot
plt.show()

By using the sample_loader and obtaining the sample.seg_data & sample.pred_data, you get two NumPy matrices on which you can compute any metric you want ;)

Can the algorithm print class-wise dice scores out for me?

The first thing that came into my mind was: No, because a Keras metric can not output multiple values. It has to be one.

But I checked StackOverflow before answering and found the obvious solution for it:
Writing a custom metric which just output the dice for a single class:

Check out the answer from Daniel Möller:
https://stackoverflow.com/questions/53926178/is-there-a-way-to-output-a-metric-with-several-values-in-keras

You can copy & paste his metric and provide it to the MIScnn Neural_Network object like this:

# Kits19: Background=0, Kidney=1, Tumor=2

model = Neural_Network(my_preprocessor, metrics=[dice_for_class(0), dice_for_class(1), dice_for_class(2)])

Then, the algorithm also prints out the class-wise dice scores, which you can store e.g. with a CSV logger callback if you want to plot some class-wise dice vs epoch figure.

Thanks for all of your help and hard work!

Thanks for the kind words!
Hope that this helps you out :)

Cheers,
Dominik