Binary neural networks, Object detection using center point detection:
Efficient Object Detection Using Binary Neural Networks,
Breno Skuk, Dimitris Milioris, Joachim Wabnig
technical report (None)
Contact: breno.skuk@gmail.com
Recent advances in the field of neural networks with quantized weights and activations down to single bit precision have allowed the development of models that can be deployed in resource constrained settings, where a trade-off between task performance and efficiency is accepted.
Here we design an efficient single stage object detector based upon the CenterNet architecture containing a combination of full precision and binarized layers. Our model is easy to train and achieves comparable results with a full precision network trained from scratch while requiring an order of magnitude less FLOP.
Even in cases where the efficient models show lower task performance than their full precision counterparts, their use is still justified by the speedup of up to 58X that comes with performing convolutions of binary weights and binary activations. This opens the possibility of deploying an object detector in applications where time is of the essence and a GPU is absent.
We study the impact of increasing the ratio of binarized MAC operations without significant increases in the total amount of MAC operations. Our study offers an insight into the the design process of binary neural networks for object detection and the involved trade-offs.
-
A decent and efficient object detector: Our implementation is straightforward and simple to understand. Although the state of the art of BNN's on object detection has almost managed to catch up with its full precision counterparts, this implementation does not involve any complicated training tricks and should be straightforward to implement.
-
Modular architecture: We modularize our network into an input module, an encoder, a decoder and a head. Each module can have it's feature maps and types of layers easily changed, allowing for experimentation with the number of network parameters and operations.
-
Create and train your own implementetion: You can test your own implementations and profit of the training pipeline on the PascalVOC dataset. The architecture folder contains an easy to modify template of a Binary CenterNet architecture that can be customized as needed.
The related paper to this implementation contains a convention thas specifies architectures modularly. The table below showcases some of the results that can be obtained by modifying the custom_qn_centernet file in the architectures folder using the convention established in the paper:
| Input | Encoder | Decoder | Heads | mAP | eq FLOP |
|---|---|---|---|---|---|
| qn(512) | qn([4 × 128, 4 × 256, 4 × 324, 4 × 512]) | tp(2)([64, 64, 32]) | h(32) | 54.6 | 1.60 |
| qn(512) | qn([2 × 128, 4 × 256, 8 × 324, 10 × 512]) | tp(4)([128, 64, 16]) | h(64) | 56.4 | 3.19 |
| qn(512) | qn([4 × 64, 4 × 128, 4 × 162, 4 × 256]) | tp(2)([64, 64, 64]) | h(32) | 40.0 | 1.53 |
Please refer to INSTALL.md for installation instructions.
You can design your own custom backbone and add it to the architectures folders.
To train using an architecture from the architectures folder:
python train.py --architecture <<architecture name>>
Here is a list of the arguments that can be given to train.py:
--architecture : Architecture that will be used to train the network.
--batch_size : Size of the batches. default=32
--num_classes : Number of classes (20 for PascalVOC). default=20
--input_size : Size of the input (eg.: 256). default=512
--gpu : Id of the GPU to use (as reported by nvidia-smi).
--epochs : Number of epochs to train. default=200
--no-snapshots : Disable saving snapshots.
--epoch-evaluation : Disable per epoch evaluation.
--final-evaluation : Disable final model evaluation.
--data-augmentation : Enable/disable data augmentation.
--model-summary : Disable model summary.
--dataset-path : Set the dataset path. default='datasets\PascalVOC-OD-2007-2012
--compute-val-loss : Disable validation loss during training.
--val-sampling : Set the fraction relative to all data to be sampled at each epoch when calculating the validation mAP during training (1 uses all val set). default=0.25
--train-sampling : Set the fraction relative to all data to be sampled at each epoch when calculating the train mAP during training (1 uses all train set). default=0.05
--train-verbose: Tensorflow .fit verbose: 0 = silent, 1 = progress bar, 2 = only epochs. default=2
For infering objects using a trained model, run:
New feature: Current version supports inference on .mp4 files. Functions can be found at utils/video_utils.py
python infer.py --architecture <<path to architecture>> --model-weights-path <<path to model weights>>
This project uses Larq, an Open-Source Library for Training Binarized Neural Networks.
We also make use of the QuickNet architecture.
Larq: An Open-Source Library for Training Binarized Neural Networks,
Lukas Geiger and Plumerai Team,
Journal of Open Source Software (https://doi.org/10.21105/joss.01746)
Objects as Points,
Xingyi Zhou, Dequan Wang, Philipp Krähenbühl,
arXiv technical report (arXiv 1904.07850)
- CenterNet + embedding learning based tracking: FairMOT from Yifu Zhang.
- Detectron2 based implementation: CenterNet-better from Feng Wang.
- Keras Implementation: keras-centernet from see-- and keras-CenterNet from xuannianz.
- MXnet implementation: mxnet-centernet from Guanghan Ning.
- Stronger human open estimation models: centerpose from tensorboy.
- TensorRT extension with ONNX models: TensorRT-CenterNet from Wengang Cao.
- CenterNet + DeepSORT tracking implementation: centerNet-deep-sort from kimyoon-young.
- Blogs on training CenterNet on custom datasets (in Chinese): ships from Rhett Chen and faces from linbior.
X
If you find this project useful for your research, please use the following BibTeX entry.
@inproceedings{XXX,
title={Efficient Object Detection using Binary Networks},
author={Skuk, Breno and Milioris, Dimitris and Wabnig, Joachim},
booktitle={XX},
year={2022}
}