Xrayto3DShape: A Benchmark for Biplanar X-rays to 3D Shape Reconstruction
Paper · Benchmark . Code Example

Table of Contents

1. Introduction
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Benchmark Results
5. Model Zoo
6. Community
7. License
8. Acknowledgments

News and Updates

• [02/09/2024] We have released the trained models. [Trained Models]

• [02/09/2024] Benchmark evaluation logs and results of Xrayto3D benchmark has been updated! [Results] [wandb]

• [12/20/2023] The Xrayto3D benchmark paper has been accepted at NeurIPS 2023 Dataset and Benchmark Track! [Openreview]

Xrayto3D-benchmark

This is a pytorch-based Python package for Biplanar X-ray to 3D Shape Reconstruction. We aim to become a comprehensive benchmarking tool for developing and evaluating methods and architectures specific to this problem.

Currently, we provide tools to train and evaluate on 4 different bone anatomies, using publicly available CT-segmentation datasets. We also define tasks for domain shifts to evaluate the robustness of these methods.

Various Encoder-Decoder Architectures on volumetric grid-based representation are reimplemented and benchmarked.

Getting Started

---

Prerequisites

To install the required packages

conda create env --name xrayto3dbenchmark-cuda_11 --f  benchmark-requirements-cuda_11.yaml
# or
conda create env --name xrayto3dbenchmark-cuda_10 --f benchmark-requirements-cuda_10.yaml
conda activate ...

Install the XrayTo3DShape package

pip install XrayTo3DShape  

Sample Training script call

python train.py  configs/paths/femur/30k/TotalSegmentor-femur-left-DRR-30k_train+val.csv configs/paths/femur/30k/TotalSegmentor-femur-left-DRR-30k_test.csv --gpu 0 --tags model-compare --size 128 --batch_size 4 --accelerator gpu --res 1.0 --model_name MultiScale2DPermuteConcat --epochs -1 --loss DiceLoss  --lr 0.002 --steps 4000 --dropout

Prepare Datasets

A separate repo for preprocessing data has been created here. The instructions for downloading and processing datasets are described here.

Usage

---

Examples to help you get familiar with the XrayTo3DShape package for quick use, evaluate an existing architecture on your own dataset, or benchmark new architectures.

Quick Start

• Beginning Examples
  • Sample Model Prediction
• Customize Datasets
• Regarding Orientation alignment between Input X-rays and Output Segmentation

Training

See bash_scripts dir

Evaluation

• Domain Shift
• Angle Perturbation

Benchmark Results

wandb results page

Model Zoo

TODO: Add pre-trained models

Acknowledgements

We took considerable inspiration and references from:

• USB
• MONAI

Various adaptation of code(or code snippets) from these sources:

• FracReconNet
• VerSe
• torch-template

Various sections of the code uses following external codebases:

• SpatialConfiguration-Net
• surface-distance
• TotalSegmentator