Face image analysis in dynamic scenarios

Abstract

Automatic personality analysis using computer vision is a relatively new research topic. It investigates how a machine could automatically identify or synthesize human person ality. Utilizing time-based sequence information, numerous attempts have been made to tackle this problem. Various applications can benefit from such a system, including pre screening interviews and personalized agents. In this thesis, we address the challenge of estimating the Big-Five personality traits along with the job candidate screen ing variable from facial videos. We proposed a novel frame work to assist in solving this challenge. This framework is based on two main components: (1) the use of Pyramid Multi level (PML) to extract raw facial textures at different scales and levels; and (2) the extension of the Covariance Descriptor (COV) to combine several local texture features of the face image, such as Local Binary Patterns (LBP), Local Directional Pattern (LDP), Binarized Statistical Image Features (BSIF), and Local Phase Quantization (LPQ). The video stream fea tures are then represented by merging the face feature vectors, where each face feature vector is formed by concatenating all iii iii the PML-COV feature blocks. These rich low-level feature blocks are obtained by feeding the textures of PML face parts into the COV descriptor. The state-of-the-art approaches are even hand-crafted or based on deep learning. The Deep Learning methods perform better than the hand-crafted descriptors, but they are com putationally and experimentally expensive. In this study, we compared five hand-crafted methods against five methods based on deep learning in order to determine the optimal balance between accuracy and computational cost. The ob tained results of our PML-COV framework on the ChaLearn LAP APA2016 dataset compared favourably with the state-of the-art approaches, including deep learning-based ones. Our future aim is to apply this framework to other similar com puter vision problem