Exploration meets exploitation: Multitask learning for emotion recognition based on discrete and dimensional models

Discrete and dimensional models can be applied to emotion recognition tasks in both psychology and computer science. Both models are advantageous. Specifically, in dimensional models, the differences and similarities between emotions can be easily estimated because of the continuity of numerical vectors used for their representation. In discrete models, core emotions usually have better interpretations, which biologically determine the emotional responses, and their expression and recognition are the same for all individuals regardless of cultural or ethnic differences. However, the problem of building a cooperative mechanism by integrating the advantages of these two models has been ignored for a long time. Therefore, in this work, we introduce an exploration and exploitation mechanism to address this problem. This mechanism not only enables the emotion recognition model to accurately locate simple and discrete emotional anchors in the entire continuous emotion space (exploration) but also encourages it to effectively search for complex and subtle emotional states near the emotional anchors (exploitation). Specifically, we propose a multitask graph neural network (MGNN) to implement this mechanism, introduce a weighting scheme to better balance the exploration and exploitation according to the task-dependent uncertainty, and filter out the key contextual features in different tasks by applying a self-attention mechanism. The experimental results show that our MGNN outperforms advanced methods on the tested datasets. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

Both discrete and dimensional models can be applied to emotion recognition tasks, and in this work, a mechanism combining the advantages of these two models is introduced. The multitask graph neural network (MGNN) is proposed to implement this mechanism, balancing exploration and exploitation through a weighting scheme and self-attention mechanism. Experimental results show the superiority of MGNN over advanced methods on tested datasets.