DNN-Boosted Information Discovery

By Luca La Fisca and Baptiste Piron, Faculté Polytechnique, UMONS

Introduction

This project is based on Docker. The following points explain how to install, run and compute the output of the multiple networks implemented. In addition to the main objective, the attention map, this docker has also several purposes.

• SSD: Single Shot Multibox Detector. Original paper. Tensorflow implementation
• Mask R-CNN. Original paper. Keras & Tensorflow implementation.
• CTPN: Connectionist Text Proposal Network. Original Paper. Tensorflow implementation
• Face detector from dlib.
• Face landmark from dlib.
• Face detection with OpenCV
• RetinaNet (objects and faces). Original paper. Keras implementation.

The attention map is established from the output of the previous networks. To increase the flexibility, the user can switch between multiple networks and therefore increase the accuracy or reduces the run time. Indeed, in the case of object detection, Mask R-CNN provides a better accuracy than SSD with a higher run time.

Requirements

This project is based on Docker. There is no external tools needed for the web page access. However, if the user intends to use the Python client, the following librairies must be installed.

• OpenCV
• requests
• requests_toolbelt
• numpy
• csv

Weights files are available on Mega.

Installation

Access to the docker folder and create a new environment with the following command.

docker build -t infoDiscovery .

Starting

Run the previously compiled docker. Mapped port can be edited to match the user configuration. In this case, the port is 80.

docker run -it -p 80:8000 infoDiscovery

Computing

From there, there are multiple ways to access the network to compute the expected output. As it is a HTTP server, the user could use a classical web page. For more advanced purposes, a Python client is provided to compute larger amount of data.

Web page

This is the easiest approach. Once the docker is run, just access to the index page.

http://localhost/

According to the operating system, the user might have to replace the localhost into the virtual machine IP. To find this IP, execute the following command:

docker-machine ip Default 

Python client

This method allows the user to select a whole folder as an input. The Python client expects a minimum of 3 arguments:

• ID of the network, which can be found in the get_network_path function of client.py.
• path of source images
• path of destination

In extension to those 3 mandatory arguments, the user can also specify two optionnal arguments:

• docker IP. Default IP is localhost.
• docker port. Default port is 80.

As explained in the introduction, the client allows to access to each network in a more convenient way than the web page. In extension, it also offers the possibility to make a csv file from source images. This CSV will allows the user to make a research among a huge database in the following way:

• class of the object.
• occurence of this class.
• position of this (those) object(s).

For example, this tool provides an easy way to search pictures with a car on the top right of the image or with 4 peoples at the left of the image. While it is not directly implemented, the user could use this tool several times to make advanced filtering.