Setup application

IRCGN-head-compare application

Table of contents

• Built with
• Getting Started
  • Dependencies Installation
  • CUDA
  • Libraries
• Services
  • All services
  • Pipeline
• Usage
  • Extract faces
  • Get faces feature
    • Lips opening
    • Euler angles
    • Expression
    • Masks
  • Transform all features to CSV
  • Train decoder
  • Generate image
• Development and source code
  • File structure
  • To improve

Built with

• Python - Python is a programming language that lets you work quickly and is great for deep learning
• Tensorflow - TensorFlow is an open source machine learning tool developed by Google
• Keras - Keras is an open-source neural-network library written in Python, it is capable of running on top of Tensorflow
• OpenCV - Keras is an open-source neural-network library written in Python, it is capable of running on top of Tensorflow
• Doxygen - Doxygen is a free licensed documentation generator capable of producing software documentation from the source code of a program

Getting started

Get the IRCGN-head-compare source code :

git clone https://github.com/LafLaurine/imac2-memoire-ircgn

Dependencies installation

Python 3.7 or higher

sudo apt-get install python3.7

Pip

sudo apt-get install mesa-utils python3-pip build-essential

To install Python3.7 for windows you have to download it from the official website : https://www.python.org/downloads/. Then, to configure it and use pip, refer to these instructions.

Numpy 1.7 or higher

pip install numpy

OpenCV 3.0 or higher

sudo apt-get install python3-opencv

or

pip install opencv-python

And if you want to have some video codecs

pip install ffmpeg ffmpeg-python
pip install gstreamer-player

Matplotlib

sudo apt-get install python3-matplotlib

or

pip install matplotlib

Pandas

pip install pandas

Pillow

pip install pillow

PyTorch 1.0 or higher with CUDA (see CUDA installation below)

Guided download link : https://pytorch.org/get-started/locally/

pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html 

PyTorch without CUDA

pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html 

Tensorflow with CUDA

pip install tensorflow-gpu==1.15

Tensorflow without CUDA (see CUDA installation below)

pip install tensorflow==1.15

For optimal performance it is highly recommended to run the code using a CUDA enabled GPU, which involved to install Cuda and CuDNN (you must have an NVIDIA graphics card).

CUDA

10.1 (Ubuntu 19.04)

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get install dkms build-essential
sudo apt-get update
sudo apt-get install nvidia-driver-450
sudo ubuntu-drivers autoinstall

_After the driver install go ahead and reboot_
sudo shutdown -r Now

_Install CUDA dependencies_
sudo apt-get install freeglut3 freeglut3-dev libxi-dev libxmu-dev

sudo apt install nvidia-cuda-toolkit
nvcc --version

CUDA for all platforms

Guided download link: https://developer.nvidia.com/cuda-downloads

cuDNN

Guided download link: https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html

Libraries

Face-alignment written by Bulat Adrian and Tzimiropoulos Georgios sudo apt install nvidia-cuda-toolkit gcc-6

pip install face-alignment

Face-expression-recognition written by Jie Wu

It doesn't need a specific installation except dependencies above (pytorch)

Services

Quick overview of all implemented services. Each services works on his own, so if you only want to extract faces of an input video, you can.

All services

Extract faces : extract faces from a video

Get faces feature : get face orientation (Euler's angles), lips opening, expression and masks of each faces contained in a directory

Transform all features to CSV : put these feature into a CSV for neural network training purpose

Train decoder : train a neural network for our decoder

Generate image : use the decoder to generate an image with given parameter

Pipeline

Usage of the services

Extract faces

Get faces feature

Lips opening

Euler's angles

Expression

Masks

Transform all features to CSV

Train decoder

Generate image