Journal
COMPUTER VISION AND IMAGE UNDERSTANDING
Volume 140, Issue -, Pages 83-92Publisher
ACADEMIC PRESS INC ELSEVIER SCIENCE
DOI: 10.1016/j.cviu.2015.07.005
Keywords
Facial expression recognition; Local texture descriptor; Local shape descriptor; Multimodal fusion
Funding
- National Natural Science Foundation of China (NSFC) [11401464, 61202237, 61273263, 61303121]
- China Postdoctoral Science Foundation [2014M560785]
- Specialized Research Fund for the Doctoral Program of Higher Education [20121102120016]
- Research Program of State Key Laboratory of Software Development Environment [SKLSDE-2015ZX-30]
- French research agency, Agence Nationale de la Recherche (ANR) [ANR-07-SESU-004, ANR-2010-INTB-0301-01, ANR-13-INSE-0004-02]
- Ecoles Centrales
- Beihang University
- Fundamental Research Funds for the Central Universities
Ask authors/readers for more resources
We present a fully automatic multimodal 2D + 3D feature-based facial expression recognition approach and demonstrate its performance on the BU-3DFE database. Our approach combines multi-order gradient-based local texture and shape descriptors in order to achieve efficiency and robustness. First, a large set of fiducial facial landmarks of 20 face images along with their 3D face scans are localized using a novel algorithm namely incremental Parallel Cascade of Linear Regression (iPar-CLR). Then, a novel Histogram of Second Order Gradients (HSOG) based local image descriptor in conjunction with the widely used first-order gradient based SIFT descriptor are used to describe the local texture around each 2D landmark. Similarly, the local geometry around each 3D landmark is described by two novel local shape descriptors constructed using the first-order and the second-order surface differential geometry quantities, i.e., Histogram of mesh Gradients (meshHOG) and Histogram of mesh Shape index (curvature quantization, meshHOS). Finally, the Support Vector Machine (SVM) based recognition results of all 2D and 3D descriptors are fused at both feature-level and score-level to further improve the accuracy. Comprehensive experimental results demonstrate that there exist impressive complementary characteristics between the 2D and 3D descriptors. We use the BU-3DFE benchmark to compare our approach to the state-of-the-art ones. Our multimodal feature-based approach outperforms the others by achieving an average recognition accuracy of 86.32%. Moreover, a good generalization ability is shown on the Bosphorus database. (C) 2015 Elsevier Inc. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available