Skip to content

Commit de46d01

Browse files
ShivamDuggal4ShivamDuggal4
committed
deeppruner readme uploaded
1 parent 3d18396 commit de46d01

File tree

3 files changed

+157
-1
lines changed

3 files changed

+157
-1
lines changed

‎deeppruner/README.md

+156
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# DeepPruner
2+
3+
This is the code repository for **DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch**.
4+
![](../readme_images/DeepPruner.png)
5+
6+
7+
##### Table of Contents
8+
[Requirements](#Requirements)
9+
[Training and Evaluation](#TrainingEvaluation)
10+
- [KITTI](#KITTI)
11+
- [Sceneflow](#Sceneflow)
12+
- [Robust Vision Challenge](#ROB)
13+
14+
<a name="Requirements"></a>
15+
## Requirements
16+
17+
+ Pytorch (0.4.0+)
18+
+ python (2.7)
19+
+ scikit-image
20+
+ tensorboardX
21+
+ torchvision (0.2.0+)
22+
23+
<a name="TrainingEvaluation"></a>
24+
## Training and Evaluation
25+
26+
**NOTE:** We allow the users to modify a bunch of model parameters and training setting for their own purpose. You may need to retain the model with the modified parameters.
27+
Check **'models/config.py'** for more details.
28+
29+
<a name="KITTI"></a>
30+
### KITTI Stereo 2012/2015:
31+
KITTI 2015 has 200 stereo-pairs with ground truth disparities. We used 160 out of these 200 for training and the remaining 40 for validation. The training set was further augmented by 194 stereo image pairs from KITTI 2012 dataset.
32+
33+
#### Setup:
34+
1. Download the KITTI 2012 and KITTI 2015 datasets.
35+
2. Split KITTI Stereo 2015 training dataset into "training" (160 pairs) and "validation" (40 pairs), following the same directory structure as of the original dataset. **Make sure to have the following structure**:
36+
> training_directory_stereo_2015 \
37+
> |----- image_2 \
38+
> |----- image_3 \
39+
> |----- disp_occ_0 \
40+
> val_directory_stereo_2015 \
41+
> |----- image_2 \
42+
> |----- image_3 \
43+
> |----- disp_occ_0 \
44+
> train_directory_stereo_2012 \
45+
|----- colored_0 \
46+
|----- colored_1 \
47+
|----- disp_occ \
48+
> test_directory \
49+
|----- image_2 \
50+
|----- image_3
51+
3. Note that, any other dataset could be used for training, validation and testing. The directory structure should be same.
52+
53+
#### Training Command:
54+
1. Like previous works, we fine-tuned the pre-trained Sceneflow model on KITTI dataset.
55+
2. Training Command:
56+
> python finetune_kitti.py \\\
57+
> --loadmodel <path_to_sceneflow_model> \\\
58+
> --savemodel <path_to_save_directory_for_trained_models> \\\
59+
> --train_datapath_2015 <training_directory_stereo_2015> \\\
60+
> --val_datapath_2015 <val_directory_stereo_2015> \\\
61+
> --datapath_2012 <directory_stereo_2012>
62+
>
63+
3. Training command arguments:
64+
+ --loadmodel (default: None): If not set, the model would train from scratch.
65+
+ --savemodel (default: './'): If not set, the script will save the trained models after every epoch in the same directory.
66+
+ --train_datapath_2015 (default: None): If not set, KITTI stereo 2015 dataset won't be used for training.
67+
+ --val_datapath_2015 (default: None): If not set, the script would fail. The validation dataset should have atleast one image to run.
68+
+ --datapath_2012 (default: None): If not set, KITTI stereo 2012 dataset won't be used for training.4. Training and validation tensorboard runs will be saved in **'./runs/' directory**.
69+
70+
#### Evaluation Command:
71+
1. We used KITTI 2015 Stereo testing set for evaluation. (Note any other dataset could be used.)
72+
2. Evaluation command:
73+
> python finetune_kitti.py \
74+
> --loadmodel <path_to_trained_model> \
75+
> --save_dir <director_to_store_disparity_output> \
76+
> --datapath <test_directory>
77+
78+
#### Metrics/ Results:
79+
1. The metrics used for evaluation are same as provided by the [KITTI Stereo benchmark](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=stereo).
80+
2. The quantitaive results obtained by DeepPruner are as follows: (Note the standings in the Tables below are at the time of March 2019.)
81+
82+
<p align="center">
83+
<img src="../readme_images/kitti_results.png" width="60%" />
84+
</p>
85+
86+
3. Alongside learning the disparity (or depth maps), DeepPruner is able to predict the uncertain regions (occluded regions , bushy regions, object edges) efficiently . Since the uncertainty in prediction correlates well with the error in the disparity maps (Figure 7.), such uncertainty can be used in other downstream tasks.
87+
88+
4. Qualitative results are as follows:
89+
90+
91+
![](../readme_images/KITTI_test_set.png)
92+
![](../readme_images/CRP.png)
93+
![](../readme_images/uncertainty_vis.png)
94+
95+
96+
<a name="Sceneflow"></a>
97+
### Sceneflow:
98+
99+
#### Setup:
100+
1. Download [Sceneflow dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html#downloads), which consists of FlyingThings3D, Driving and Monkaa (RGB images (cleanpass) and disparity).
101+
102+
2. We followed the same directory structure as of the downloaded data. Check **dataloader/sceneflow_collector.py**.
103+
104+
105+
106+
#### Training/ Evaluation Command:
107+
> python train_sceneflow.py \\\
108+
> --loadmodel <path_to_trained_model> \\\
109+
> --save_dir <directory_to_store_disparity_output> \\\
110+
> --savemodel <directory_to_store_trained_models_every_epoch> \\\
111+
> --datapath_monkaa <monkaa_dataset> \\\
112+
> --datapath_flying <flying_things> \\\
113+
> --datapath_driving <driving_dataset>
114+
115+
#### Metrics/ Results:
116+
1. We used EPE(end point error) as one of the metrics.
117+
118+
<p align="center">
119+
<img src="../readme_images/sceneflow.png" width="60%" />
120+
</p>
121+
122+
<p align="center">
123+
<img src="../readme_images/sceneflow_results.png"/>
124+
</p>
125+
126+
127+
128+
129+
130+
<a name="ROB"></a>
131+
### Robust Vision Challenge:
132+
133+
#### Details:
134+
1. The goal of Robust Vision Challenge challenge is to foster the development of vision systems that are robust and
135+
consequently perform well on a variety of datasets with different characteristics.
136+
Please refer to <a href="http://www.robustvision.net/"> Robust Vision Challenge </a> for more details.
137+
138+
2. We used the pre-trained Seneflow model and then jointly fine-tuned the model on KITTI, ETH3D and Middlebury datasets.
139+
140+
#### Setup
141+
1. Dataloader and setup details coming soon.
142+
143+
#### Metrics/ Results:
144+
145+
Check <a href="http://www.cvlibs.net/datasets/kitti/eval_scene_flow_detail.php?benchmark=stereo&result=d67c3b20f7d2724e8d6a25f49405c0511f883b7a">DeepPruner_ROB</a> on KITTI benchmark. \
146+
Check <a href="https://www.eth3d.net/low_res_two_view">DeepPruner_ROB</a> on ETH3D benchmark. \
147+
Check <a href="http://vision.middlebury.edu/stereo/eval3/">DeepPruner_ROB</a> on MiddleburyV3 benchmark. \
148+
149+
<p align="center">
150+
<img src="../readme_images/rob.png" width="60%" />
151+
</p>
152+
153+
<p align="center">
154+
<img src="../readme_images/rob_results.png" width="60%"/>
155+
</p>
156+

‎deeppruner/models/config.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ def __init__(self, d):
2626

2727

2828
config = {
29-
"max_disp": 192, #*2 for using DeepPruner-fast pre-trained model,
29+
"max_disp": 192,
3030
"cost_aggregator_scale": 4, # for DeepPruner-fast change this to 8.
3131
"mode": "training", # for evaluation/ submission, change this to evaluation.
3232

‎readme_images/rob_results.png

79.6 KB
Loading

0 commit comments

Comments
 (0)