TITAN-Net introduces a fresh, effective approach for bridging the gap between sensor modalities with different data formats! 🌉 By harnessing the power of scene semantics, TITAN-Net can, for the very first time, synthesize a panoramic color image directly from a 3D LiDAR point cloud.
- Semantic Segmentation: First, we segment the LiDAR point cloud and project it onto a spherical surface. 📡
- Modular Generative Framework: Our approach is modular and generative, translating LiDAR segments into colorful panoramic images!
- Adversarial Learning: Using a conditional GAN, we translate LiDAR segment maps to their camera image counterparts, creating a seamless color scene. 🎨
- Performance: Quantitative evaluations on the Semantic-KITTI dataset show TITAN-Net outperforms strong baselines by a significant margin.
Below are links to the models and pretrained weights used in this project:
- TITAN-Net Weights
- SD-Net Weights — from NVIDIA's semantic-segmentation repository
- SalsaNext Weights — from Tiago Cortinhal's SalsaNext repo
- TITAN-Next Weights — from Tiago Cortinhal's TITAN-Next repository
Check out these example videos showing TITAN-Net in action, generating breathtaking RGB panoramic images! 🎥
See how easily we can use semantic segmentation maps for data augmentation in datasets like KITTI and Cityscapes!
If you use TITAN-Net in your research, please consider citing our paper:
@inproceedings{cortinhal2021semanticsaware,
title={Semantics-Aware Multi-Modal Domain Translation: From LiDAR Point Clouds to Panoramic Color Images},
author={Cortinhal, Tiago and Kurnaz, Fatih and Aksoy, Eren Erdal},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages={3747-3756},
year={2021}
}