The repository contains the evaluation workflows for the BraTS 2023 challenge and beyond, including:
- BraTS 2023
- BraTS-GoAT 2024
- FeTS 2024
- BraTS 2024
Workflows are organized by task type, e.g. the wf-augmentation folder contains
the CWL workflows for the Augmentation task. CWL scripts used by 1+ workflow
are located in the shared folder.
Source code of the metrics computations mentioned in the README are available
in the evaluation folder of this repo, organized into sub-folders by task.
Branch: main
BraTS 2024 is an extension to BraTS 2023, and will also follow the two evaluation phases approach.
Metrics returned and used for ranking will depend on the task:
| Task | Metrics | Ranking |
|---|---|---|
| Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
| Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics |
| Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index |
| Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics |
Branch: brats2023
BraTS 2023 is split into two evaluation phases:
-
Validation phase: participants submit predictions files (segmentation masks, t1n inferences, etc.) to be evaluated using the validation dataset
-
Test phase: participants submit MLCube models that will generate prediction files using the test dataset
Metrics returned and used for ranking will depend on the task:
| Task | Metrics | Ranking |
|---|---|---|
| Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
| Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE |
| Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance |
Branch: brats_goat2024
Similar to BraTS 2023, BraTS-GoAT 2024 is split into two evaluation phases:
-
Validation phase: participants submit segmentation predictions to be evaluated using the validation dataset
-
Test phase: participants submit MLCube models that will generate segmentation predictions using the test dataset
Metrics returned and used for ranking are:
| Metrics | Ranking |
|---|---|
| Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
Branch: fets2024
FeTS 2024 has one evaluation phase facilitated by this repo:
- Validation phase: participants submit segmentation predictions to be evaluated using the validation dataset
Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity
The Code submission phase is handled by the FeTS-AI Task 1 infrastructure.
BraTS 2023+ evaluation would not be possible without the work of:
- @FelixSteinbauer - inpainting metrics
- @rachitsaluja - lesionwise segmentation metrics
- @sarthakpati - pathology metrics
In addition to: