This repository contains refactored code for "Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching" by Stepan Tulyakov, Anton Ivanov and Francois Fleuret, that appeared on NeurIPS2018 as a poster.
Please install conda with python 3.6 and pytorch 1.0. Next install all dependencies by running
conda install --yes --file requirements.txt
To set up FlyingThings3D dataset, download PNG RGB cleanpass images and disparities from website of Patter Recognition and Image Processing group of University of Freiburg. Unpack the archive with images into PracticalDeepStereo_NIPS2018/datasets/flyingthings3d/frames_cleanpass and archive with disparities into PracticalDeepStereo_NIPS2018/datasets/flyingthings3d/disparity.
To run training on Flyingthings3D run
./train_on_flyingthings3d.py \
--experiment_folder experiments/flyingthings3d \
--dataset_folder datasets/flyingthings3d \
During the first run, the dataset object calculates and saves disparity statistic for every example in the dataset. Therefore, it might take a while before actual training starts. Overall, the training on full-size image is very slow since it does not use batch processing. The training can be started from a checkpoint by setting the --checkpoint_file flag.
To benchmark on Flyingthings3D run
./benchmark_on_flyingthings3d.py \
--experiment_folder experiments/flyingthings3d \
--dataset_folder datasets/flyingthings3d \
--checkpoint_file experiments/flyingthings3d/010_checkpoint.bin \
--is_psm_protocol
The evalutaion protocol can be selected by setting / unsetting the --is_psm_protocol flag.
Pretrained model with training plot and log are now avaliable.
For the pre-trained model test results are following
| Protocol | MAE, [pix] | 3PE, [%] | time-per-image, [sec] |
|---|---|---|---|
| PSM | 1.04 | 3.09 | 0.62 |
| CRL | 0.98 | 2.96 |
Note, that this result is slightly better than the one reported in the paper according to the PSM protocol and slightly worse according to CRL (but both results are still very good!).
If one of the training scripts does not work please run all unit tests by executing ./run_unit_tests.sh. This will help you to localize and fix bugs on your own.