Access here. Citation:
Shangke Liu, Mohamed Amgad, Deeptej More, Muhammad A. Rathore, Roberto Salgado, Lee A. D. Cooper: A panoptic segmentation dataset and deep-learning approach for explainable scoring of tumor-infiltrating lymphocytes
npj Breast Cancer 10, 52 (2024). https://doi.org/10.1038/s41523-024-00663-1
Tumor-Infiltrating Lymphocytes (TILs) have strong prognostic and predictive value in breast cancer, but their visual assessment is subjective. To improve reproducibility, the International Immuno-oncology Working Group recently released recommendations for the computational assessment of TILs that build on visual scoring guidelines. However, existing resources do not adequately address these recommendations due to the lack of annotation datasets that enable joint, panoptic segmentation of tissue regions and cells. Moreover, existing deep-learning methods focus entirely on either tissue segmentation or cell nuclei detection, which complicates the process of TILs assessment by necessitating the use of multiple models and reconciling inconsistent predictions. We introduce PanopTILs, a region and cell-level annotation dataset containing 814,886 nuclei from 151 patients, openly accessible at: sites.google.com/view/panoptils. Using PanopTILs we developed MuTILs, a neural network optimized for assessing TILs in accordance with clinical recommendations. MuTILs is a concept bottleneck model designed to be interpretable and to encourage sensible predictions at multiple resolutions. Using a rigorous internal-external cross-validation procedure, MuTILs achieves an AUROC of 0.93 for lymphocyte detection and a DICE coefficient of 0.81 for tumor-associated stroma segmentation. Our computational score closely matched visual scores from 2 pathologists (Spearman R = 0.58–0.61, p < 0.001). Moreover, computational TILs scores had a higher prognostic value than visual scores, independent of TNM stage and patient age. In conclusion, we introduce a comprehensive open data resource and a modeling approach for detailed mapping of the breast tumor microenvironment.
Use a containerized environment to run and train MuTILs.
MuTILs has been developed as a part of cTME (Computational pipelines for analysis of Tumor MicroEnvironment) project for which there is a publicly available Docker image. For convenience, follow the steps below to set up the environment and make an inference with MuTILs.
Clone this repository then pull the container on your GPU server.
git clone https://github.com/PathologyDataScience/MuTILs_Panopticcd MuTILs_Panopticgit submodule update --init --recursivedocker pull kheffah/ctme
The code is built on Python 3.8 and the container environment hosts a 10.2 CUDA on a 18.04 Ubuntu system.
Modify both files of
MuTILs_Panoptic/run_container.shMuTILs_Panoptic/configs/MuTILsWSIRunConfigs.yaml
with the correct paths of files and folders on your system.
./run_container.shcd /home/mtageld/Desktop/MuTILs_Panoptic/utils/CythonUtilspython setup.py build_ext --inplacecd /home/mtageld/Desktop/
Within the container, run the MuTILsWSIRunner.py module to perform inference on your set of slides at the location defined in the configuration YAML file.
python MuTILs_Panoptic/mutils_panoptic/MuTILsWSIRunner.py
Note: Do not forget to give permission to your folders to make them accessible for MuTILs.
Please, contact the authors to request access to the model weights.