MooneyFaceGenerator

A Deep Learned Model for Generating Mooney Faces from Face Dataset

Mooney Face Classification and Prediction by Learning across Tone

Tsung-Wei Ke, Stella X. Yu, David Whitney

Mooney faces are special two-tone image, and researchers believe that these images might contain essential element of facial structure which helps huma to percept faces. However, researcher are also bothered with two issues: 1) only small number of Mooney faces available, 2) source photos of these Mooney images are lost.

To address these issues, we propose two models:

1. Mooney faces generator
2. Binary-to-Grayscale images predictor

We provide source code of Mooney faces generator in this repository, and we train a Pix2Pix GAN as Binray-to-Grayscale images predictor.

Mooney faces generator

Prerequisites

• Linux
• NVIDIA GPU + CUDA +CUDNN

Getting Started

• Install torch and dependencies
• Install torch packages cudnn, dpnn, hdf5

> luarocks install cudnn
> luarocks install dpnn
> luarocks install hdf5

• Clone this repo

> git clone https://github.com/buttomnutstoast/MooneyFaceGenerator.git
> cd MooneyFaceGenerator

Prepare the datasets

Put the downloaded datasets to $DATA_ROOT/. The folder structure should be in the format like

DATA_ROOT/
    |------FaceScrub/
    |         |--------Img/
    |         |--------img_list.txt
    |
    |------ILSVRC/
    |         |--------Img/
    |         |--------revImg/
    |         |--------img_list.txt
    |         |--------rev_img_list.txt
    |
    |------Mooney/
    |         |--------Img/
    |         |--------revImg/
    |         |--------vflipImg/
    |         |--------vflipRevImg/
    |         |--------img_list.txt
    |         |--------rev_img_list.txt
    |         |--------vflip_img_list.txt
    |         |--------vflip_rev_img_list.txt
        ....

• Download Cropped and Aligned Facescrub Dataset provided by MegaFace

> Need to be finished

• Download ImageNet

> Need to be finished

• Download Our Mooney Groun-Truth Data

> cd $DATA_ROOT
// down load from https://www.dropbox.com/s/rkogofo9pz92vla/Mooney.tar?dl=0
> tar xvf Mooney.tar

Train the Mooney face classifier

We first fine-tune the facial recognition model by openface for grayscale face classification. You can download the model from here and then fine-tune it to mooney face classifier. Put the downloaded model to $TRAINED_MODEL_PATH.

> DATA_ROOT=/path/to/root/dir/of/datasets TRAINED_MODEL_PATH=/path/to/gray/face/models sh scripts/mooney_train.sh

The new learned model would be save to checkpoint/mooney_train/OPTION_ARGS/TIME_AND_DATE/model_20.t7

Generate Mooney faces

Set $TRAINED_MODEL_PATH to the trained mooney classifier (such as checkpoint/mooney_train/OPTION_ARGS/TIME_AND_DATE/model_20.t7).

> DATA_ROOT=/path/to/root/dir/of/datasets TRAINED_MODEL_PATH=/path/to/mooney/face/models sh scripts/mooney_train.sh

The results will be saved into a hdf5 file which would be located at checkpoint/facescrub_mooney/OPTION_ARGS/TIME_AND_DATE/testOutput_1.h5

Filter most-likely Mooney candidates from each images

Set $HDF5_RESULT to the filepath of hdf5 file generate in the previous step.

> HDF5_RESULT=/path/to/hdf5/result th scripts/filter_by_mooneyness.lua -hdf5 $HDF5_RESULT

Binary-to-Grayscale images predictor

Setup

Please follow the instruction and download Pix2Pix GAN from https://github.com/phillipi/pix2pix.

Dataset prepration

You can,

1. Use the faces generated from here
2. Download our Berkeley Mooney Dataset

---

Acknowledgements

We borrow code heavily from Soumith. We fine-tune the model nn4small2v1 trained by openface to mooney face classifier. We use the cropped and aligned Facescrub Dataset provided by MegaFace. Our ground-truth mooney faces are provided by David and PICS.