This repository contains my experiments with lip reading using deep learning in Keras. I train and test on the LRW dataset.
Codes to convert the videos in LRW Dataset to: - frames pertaining to words - mouth area of those frames - audio
Instructions are provided in README file in directory.
Directory to place the "shape_predictor_68_face_landmarks.dat" file, required by process-lrw and head-pose
Codes and files --- considering the lipreader as an image retrieval system
Codes and files --- to compute head pose in all frames in LRW dataset (extracted using process-lrw)
Head pose is determined using voletiv/deepgaze (my fork of deepgaze).
Instructions are provided in README file in directory.