Journal
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
Volume 13, Issue 2, Pages 713-728Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TAFFC.2019.2953664
Keywords
Kernel; Emotion recognition; Face recognition; Computer vision; Mood; Computational modeling; Group-level emotion recognition; global alignment kernels; multiple kernel learning; facial expression analysis; convolution neural network
Funding
- National Natural Science Foundation of China [62076122]
- Jiangsu SpeciallyAppointed Professor Program [3051107219003]
- Jorma Ollila Grant of Nokia Foundation
- Central Fund of the Finnish Cultural Foundation
- Natural Science Foundation of Jiangsu Higher Education Institutions of China [17KJB520010]
- Tekes Fidipro program [849/31/2015]
- Business Finland project [3116/31/2017]
- Academy of Finland
- NVIDIA Corporation
- Infotech Oulu
- Talent Startup project ofNJIT [YKJ201982]
Ask authors/readers for more resources
This article proposes a new method to effectively analyze group behavior and emotion from a group-level image, using a combination of global alignment kernels and support vector machine. The distance between two group-level images is measured using a global alignment kernel, and a global weight sort scheme is used to optimize the performance of the kernel. Experimental results demonstrate promising performance for group-level emotion recognition.
From the perspective of social science, understanding group emotion has become increasingly important for teams to considerably accomplish organizational work. Currently, automatically analyzing the perceived affect of a group of people has been received increasingly interest in affective computing community. The variability in group size makes difficulty for group-level emotion recognition to straightforwardly measure the feature distance of two group-level images. Recent works attempted to resolve the preceding problem by using feature encoding. However, the early works lack of efficiency. To alleviate this problem, this article aims to design a new method to effectively analyze the group behavior from a group-level image. Motivated by time-series kernel approaches explored in dynamic facial expression classification, this article mainly concentrates on global alignment kernel and design support vector machine with the combined global alignment kernels (SVM-CGAK) to better recognize group-level emotion. Specifically, we first propose to use global alignment kernel to explicitly measure the distance of two group-level images. For improving the performance of global alignment kernel, we use the global weight sort scheme based on their spatial relation information to sort the faces from group-level image, making an efficient data structure to the global alignment kernel. With this new global alignment kernel, we construct the backbone of SVM-CGAK, namely, support vector machine with global alignment kernel. Furthermore, considering the challenging environment, we construct two global alignment kernels based on Reisz-based Volume Local Binary Pattern and deep convolutional neural network features, respectively. Lastly, to make the robustness of group-level emotion recognition, we propose SVM-CGAK combining both global alignment kernels with multiple kernel learning approach. It can enhance the discriminative ability of each global alignment kernel. Intensive experiments are conducted on three challenging group-level emotion databases. The experimental results demonstrate that the proposed approach achieves promising performance for group-level emotion recognition compared with the recent state-of-the-art methods.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available