Material Estimation

This is an unofficial implementation of this paper for material estimation. The authors has provided their model weights in Caffe while not the code for inference(which requires denseCRF for post processing).

This repo provides codes to convert their original Caffe model to PyTorch, and re-implement three key components mentioned in the paper:

• DenseCRF
• Shift-Pooling(refer to this paper for more details)
• LRN(local response normalization in Googlenet).

Note that the denseCRF used here is RGB based and the hyper-parameters are arbitrarily copied from this repo. Please check here if you want to use the denseCRF mentioned in the paper

Requirement

• pydensecrf
• pytorch
• opencv

Model Convertion

Please use this repo to convert the Caffe model to PyTorch. After the convertion, remember to squeeze the dimension of bias(from (1,1,1,K) -> (K)) and convert the fully connected layers to convolutional layers. Code example:

# Assuming that fc6.weight.size() == (1,1,4096,25088)
f['conv_fc6.weight'] = f['fc6.weight'].squeeze().view(4096,512,7,7)
f['conv_fc6.bias'] = f['fc6.bias'].squeeze()

An example of Googlenet after conversion is provided. Note that the Alexnet provided by the authors is not usable.

Examples

Images captures from minc dataset and Ycb dataset

RGB Input	Material Segmentation

Citation

@inproceedings{bell2015material,
  title={Material recognition in the wild with the materials in context database},
  author={Bell, Sean and Upchurch, Paul and Snavely, Noah and Bala, Kavita},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={3479--3487},
  year={2015}
}

Contact and Contribution

Feel free to contact me dulucas24@gmail.com, any contribution is welcome.