About Point Pillars

Point Pillars is a very famous Deep Neural Network for 3D Object Detection for LiDAR point clouds. With the application of object detection on the LiDAR devices fitted in the self driving cars, Point Pillars focuse on fast inference ~50fps, which was magnitudes above as compared to other networks for 3D Object detection. In this repo, we are trying to develop point pillars in TensorFlow. Here's a good first post to familiarize yourself with Point Pillars.

Contributors are welcome to work on open issues and submit PRs. First time contributors are welcome and can pick up any "Good First Issues" to work on.

PointPillars in TensorFlow

Point PIllars 3D detection network implementation in Tensorflow. External contributions are welcome, please fork this repo and see the issues for possible improvements in the code.

Installation

Download the LiDAR, Calibration and Label_2 zip files from the Kitti dataset link and unzip the files, giving the following directory structure:

├── training    <-- 7481 train data
   |   ├── calib
   |   ├── label_2
   |   ├── velodyne
└── testing     <-- 7580 test data
           ├── calib
           ├── velodyne

After placing the Kitti dataset in the root directory, run the following code

git clone --recurse-submodules https://github.com/tyagi-iiitv/PointPillars.git
virtualenv --python=/usr/bin/python3.8 env
source ./env/bin/activate
pip install tensorflow-gpu tensorflow_probability sklearn opencv-python
cd PointPillars
python setup.py install
python point_pillars_training_run.py

Note: This code was tested on TF v2.5.0 (nightly at this time) but should work on all TF2 versions.

Deploy on a cloud notebook instance (Amazon SageMaker etc.)

Please read this blog article: https://link.medium.com/TVNzx03En8

Technical details about this code

Please refer to this article on Medium.

Pretrained Model

The Pretrained Point Pillars for Kitti with complete training and validation logs can be accessed with this link. Use the file model.h5.

Saving the model as .pb

Inside the point_pillars_training_run.py file, change the code as follows to save the model in .pb format.

import sys
if __name__ == "__main__":

    params = Parameters()

    pillar_net = build_point_pillar_graph(params)
    pillar_net.load_weights(os.path.join(MODEL_ROOT, "model.h5"))
    pillar_net.save('new_model')
    sys.exit()
    # This saves the model as pb in the new_model directory. 
    # Remove these lines during usual training. 

Loading the saved pb model

model = tf.saved_model.load('model_directory')