Semi-supervised parallel shared encoders for speech emotion recognition

Supervised speech emotion recognition requires a large number of labeled samples that limit its use in practice. Due to easy access to unlabeled samples, a new semi-supervised method based on autoencoders is proposed in this paper for speech emotion recognition. The proposed method performed the classification operation by extracting the information contained in unlabeled samples and combining it with the information in labeled samples. In addition, it employed maximum mean discrepancy cost function to reduce the distribution difference when the labeled and unlabeled samples were gathered from different datasets. Experimental results obtained on different emotional speech datasets demonstrated that the proposed method reached better performance than previous methods in matched, semi-matched, and mismatched conditions. As an example, the proposed method boosted the error reduction rate on the INTERSPEECH 2009 Emotion challenge task on average by 14.13% via utilizing only 200 labeled samples. Besides, the proposed method was investigated as a domain adaptation method for recognizing Persian emotional speech. In this case, the proposed method boosted the accuracy of recognition by 10% compared to that of the cross-training method when German emotional database was used as the source domain. (C) 2021 Elsevier Inc. All rights reserved.

Automated summary

This paper proposes a semi-supervised method for speech emotion recognition based on autoencoders, which combines information from unlabeled samples and labeled samples using a maximum mean discrepancy cost function to reduce distribution differences. Experimental results show that the proposed method outperforms previous methods on different emotional speech datasets, with potential for wide applications.