FaceCept3D

FaceCept3D: 3D Face Analysis and Recognition

Introduction

FaceCept3D is a realtime framework for 3D face analysis and recognition. It contains a set of extendible components that can be combined to fulfil a specific task. At this step we open source the following functionality:

1. Person-specific template creation
2. Extreme head pose estimation [coming soon]
3. Facial expression analysis [coming soon]

FaceCept3D is based on the following works:

• Robust Real-Time Extreme Head Pose Estimation. [pdf]

• Facial expression recognition under a wide range of head poses. [pdf]

Here is an example image of head pose estimation using FaceCept3D.

See FaceCept3D wiki for more details.

Requirements

To install FaceCept3D you need to have the following libraries available

• Point Cloud Library PCL version 1.7
• OpenCV >= version 2.4
• Qt version 4.8 (required by VTK 5.6)

and their dependencies.

In addition, if you plan to use a depth sensor, you will to install the driver. Currently we support only the Microsoft Kinect 1.0 sensor. However, the code can be easily extended to most of the available RGB-D sensors.

To use MS Kinect sensors:

• On windows install their SDK (v 1.8)

Installation

FaceCept3D was tested on Windows and Linux. Mac users should follow linux installation instructions.

Linux

• OpenNI and SensorKin drivers. A great guide is here.

• Prerequsities. Install the following libraries using your package manager:

  boost, eigen, flann, vtk (v 5.6), qhull, opencv, tbb, qt (v 4.8)

  For Ubuntu 14.04 the following command will do the trick:

   sudo apt-get install libboost-all-dev libeigen3-dev libflann-dev libvtk5-dev libqhull-dev libopencv-dev libtbb-dev libqt4-dev

  In addition, you need to install cmake if you don't have it:

   sudo apt-get install cmake

• Install PCL. We need to build and install pcl, since the version here is built without -std=c++11 modifier. This guide will help you build pcl from source.

• Check that everything works. Try running pcl_openni_viewier. If you don't see any output, then there is something wrong with your installation.

• Build FaceCept3D.

  git clone https://github.com/sergeytulyakov/FaceCept3D.git
  cd FaceCept3D
  mkdir build
  cd build
  cmake ..
  make

Windows

On Windows platform there are two ways of build FaceCept3D:

1. Using CMake to generate an *.sln file
2. Using the provided *.sln file.

It is somewhat more convenient to use the latter variant since the final solution file is cleaner than the one generated by CMake. If you still want to use CMake, you will need to manually enter all necessary paths.

To simplify the process we provide all the dependencies in a binary format. All libs were compiled using msvc2010-win64. You can still use it in a latter version of Visual Studio, but you need to have msvc2010 toolchain installed.

• Install Microsoft Kinect Driver 1.8. Download it here and install.

• Download and unpack precompiled dependencies. Download them here

• Set up environmental variables You may find useful Rapid Environment Editor for setting the environment on Windows.

  Set KINECTSDK10_DIR and FACECEPT3D_3RDPARTY_DIR. For example, mine are pointing to

   KINECTSDK10_DIR=c:\Program Files\Microsoft SDKs\Kinect\v1.8
   FACECEPT3D_3RDPARTY_DIR=d:\Dependencies\FaceCept3D-3rdparty\

  You also need to add the following folders to your path:

   %FACECEPT3D_3RDPARTY_DIR%\Qt\bin
   %FACECEPT3D_3RDPARTY_DIR%\PCL\bin
   %FACECEPT3D_3RDPARTY_DIR%\Opencv\bin
   %FACECEPT3D_3RDPARTY_DIR%\FLANN\bin
   %FACECEPT3D_3RDPARTY_DIR%\QHull\bin	

  The rest is configured in the Project Properties files. If something doesn't work feel free to inspect the *.props files in the Config folder. If you want to use your versions of the dependencies, you will have to change the *.props files.

  Open 3dheadposeestimation.sln using your version of Visual Studio and try to build. If you have a newer Visual Studio than 2010, do not agree to convert the solution to a newer toolchain, since the dependencies are built with msvc2010-win64 it will not link.

What to do next

Have a look at our wiki pages:

1. Getting Started
2. Point Cloud Registration
3. Personalized Template Creation with FaceCept3D

Citing FaceCept3D

If you use FaceCept3D in your research, please cite one of the following (or both):

@inproceedings{Tulyakov2014,
	author = {Tulyakov, S. and Vieriu, R. L. and Semeniuta, S. and Sebe, N.},
	booktitle = {International Conference on Pattern Recognition},
	title = {{Robust Real-Time Extreme Head Pose Estimation}},
	year = {2014},
}

@inproceedings{Vieriu2015,
	author = {Vieriu, R. L. and Tulyakov, S. and Sangineto, E. and Semeniuta, S. and Sebe, N.},
	booktitle = {Automatic Face and Gesture Recognition},
	title = {{Facial Expression Recognition under a Wide Range of Head Poses}},
	year = {2015},
}