K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human-Robot Interaction

In this article, K-means clustering-based Kernel canonical correlation analysis algorithm is proposed for multimodal emotion recognition in human-robot interaction (HRI). The multimodal features (gray pixels; time and frequency domain) extracted from facial expression and speech are fused based on Kernel canonical correlation analysis. K-means clustering is used to select features from multiple modalities and reduce dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTER-FACE'05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods without K-means clustering; more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE'05.

Automated summary

This article proposes a K-means clustering-based kernel canonical correlation analysis algorithm for multimodal emotion recognition in human-robot interaction. By fusing multimodal features from different modalities, the proposed method improves heterogeneity among modalities and enhances the accuracy of emotion recognition.