Skip to content

Latest commit

 

History

History
57 lines (48 loc) · 2.59 KB

README.md

File metadata and controls

57 lines (48 loc) · 2.59 KB
Raw

RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation

This is the implementation of the models and test code for the "RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation", ICCV2017.

File description

  • caffe-master: caffe used in our experiments
  • test.py: demo code
  • Each of NYU-50 / NYU-101 / NYU-152 directory includes RDF model and its prototxt corresponding to different number of resnet layers. (*You may need to change the 'nyud_dir' parameter in the prototxt.)
  • data: test data
  • nyud_layers.py: input python layer
  • gupta-utils-HHA: HHA generation utils by Gupta et al. [2]

Usage

  • Install Opencv
  • Compile pycaffe: modify the "Makefile.config" in caffe-master for your environment.
  • Download the model files.
  • Run test.py
    • Change 'caffe_root'
    • Set the 'scale' and 'model' to test.
    • To achieve the same accuracy reported in our paper, you need to implement multi-scale (0.6~1.2) ensemble as described in the paper.

Environment

Our experiments were mainly performed on Ubuntu 14.04 with CUDA7.0 / CUDNNv4 / Titan X (maxwell) / Opencv2.7

Note

  • Similarly to RefineNet,
    • Our implementation uses bicubic resize function to resize feature map.
    • We remove white boundaries of the images in NYUDv2.
  • Any comment for improvement is welcome as the code is not fully optimized. but please note that further maintenance will be infrequently performed.
  • OOM may occur for RDF-152 with the image scale larger than 1.0 on different environtment (e.g., Titan Xp, CUDA 8.0, CUDNN v6)

Citation

  • We would like to thank Guosheng Lin [3] for invaluable help.

[1] @InProceedings{Park_2017_ICCV, author = {Park, Seong-Jin and Hong, Ki-Sang and Lee, Seungyong}, title = {RDFNet: RGB-D Multi-Level Residual Feature Fusion for Indoor Semantic Segmentation}, booktitle = {The IEEE International Conference on Computer Vision (ICCV)}, month = {Oct}, year = {2017} }

[2] @incollection{guptaECCV14, author = {Saurabh Gupta and Ross Girshick and Pablo Arbelaez and Jitendra Malik}, title = {Learning Rich Features from {RGB-D} Images for Object Detection and Segmentation}, booktitle = ECCV, year = {2014}, }

[3] @inproceedings{lin2017refinenet, title={Refinenet: Multi-path refinement networks for high-resolution semantic segmentation}, author={Lin, Guosheng and Milan, Anton and Shen, Chunhua and Reid, Ian}, booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2017} }

License

For academic usage, the code is released under the permissive BSD license. For any commercial purpose, please contact the authors.