PVANet: Lightweight Deep Neural Networks for Real-time Object Detection

by Sanghoon Hong, Byungseok Roh, Kye-hyeon Kim, Yeongjae Cheon, Minje Park (Intel Imaging and Camera Technology) Presented in EMDNN2016, a NIPS2016 workshop (arXiv link)

Introduction

This repository is a fork from py-faster-rcnn and demonstrates the performance of PVANet.

You can refer to py-faster-rcnn README.md and faster-rcnn README.md for more information.

Desclaimer

Please note that this repository doesn't contain our in-house codes used in the published article.

• This version of py-faster-rcnn is slower than our in-house runtime code (e.g. image pre-processing code written in Python)
• PVANet was trained by our in-house deep learning library, not by this implementation.
• There might be a tiny difference in VOC2012 test results, because some hidden parameters in py-faster-rcnn may be set differently with ours.

Citing PVANet

If you want to cite this work in your publication:

@article{hong2016pvanet,
  title={{PVANet}: Lightweight Deep Neural Networks for Real-time Object Detection},
  author={Hong, Sanghoon and Roh, Byungseok and Kim, Kye-Hyeon and Cheon, Yeongjae and Park, Minje},
  journal={arXiv preprint arXiv:1611.08588},
  year={2016}
}

Installation

1. Clone the Faster R-CNN repository

   # Make sure to clone with --recursive
   git clone --recursive https://github.com/sanghoon/pva-faster-rcnn.git

2. We'll call the directory that you cloned Faster R-CNN into FRCN_ROOT. Build the Cython modules

   cd $FRCN_ROOT/lib
   make

3. Build Caffe and pycaffe

   cd $FRCN_ROOT/caffe-fast-rcnn
   # Now follow the Caffe installation instructions here:
   #   http://caffe.berkeleyvision.org/installation.html
   # For your Makefile.config:
   #   Uncomment `WITH_PYTHON_LAYER := 1`
   
   cp Makefile.config.example Makefile.config
   make -j8 && make pycaffe

4. Download PVANet detection model for VOC2007

   cd $FRCN_ROOT
   ./models/pvanet/download_voc2007.sh

5. Download PVANet detection model for VOC2012 (published model)

   cd $FRCN_ROOT
   ./models/pvanet/download_voc_best.sh

6. (Optional) Download all available VOC models (including pre-trained and compressed models)

   cd $FRCN_ROOT
   ./models/pvanet/download_all_models.sh

7. (Optional) Download ImageNet classification model

   cd $FRCN_ROOT
   ./models/pvanet/download_imagenet_model.sh

How to run the demo

1. Download PASCAL VOC 2007 and 2012

• Follow the instructions in py-faster-rcnn README.md

2. PVANet on PASCAL VOC 2007

   cd $FRCN_ROOT
   ./tools/test_net.py --net models/pvanet/pva9.1/PVA9.1_ImgNet_COCO_VOC0712.caffemodel --def models/pvanet/pva9.1/faster_rcnn_train_test_21cls.pt --cfg models/pvanet/cfgs/submit_1019.yml --gpu 0

3. PVANet (compressed)

   cd $FRCN_ROOT
   ./tools/test_net.py --net models/pvanet/pva9.1/PVA9.1_ImgNet_COCO_VOC0712plus_compressed.caffemodel --def models/pvanet/pva9.1/faster_rcnn_train_test_ft_rcnn_only_plus_comp.pt --cfg models/pvanet/cfgs/submit_1019.yml --gpu 0

Expected results

Mean Average Precision on VOC detection tasks

Model	VOC2007 mAP (%)	VOC2012 mAP (%)
PVANet+ (VOC2007)	84.9	N/A
PVANet+ (VOC2012)	89.8	84.2
PVANet+ (VOC2012 + compressed)	87.8	83.7

• The training set for the VOC2012 model includes the VOC2007 test set. Therefore the accuracies on VOC2007 of the model are not meaningful; They're shown here just for reference

Validation error on ILSVRC2012

Input size	Top-1 error (%)	Top-5 error (%)
192x192	30.00	N/A
224x224	27.66	8.84

• We re-trained a 224x224 model from the '192x192' model as a base model.