Journal
INFORMATION FUSION
Volume 76, Issue -, Pages 422-428Publisher
ELSEVIER
DOI: 10.1016/j.inffus.2020.11.007
Keywords
Emotion recognition; Loss function; Multi-task machine learning; Deep learning; Unbalanced data
Funding
- Polytechnic University of Haut de France
- Haut de France region
Ask authors/readers for more resources
This paper proposes a new deep learning architecture for context-based multi-label multi-task emotion recognition, with a key focus on the new loss function called multi-label focal loss (MFL). Experimental results demonstrate that the combination of MFL with Huber loss performs the best, outperforming other combinations of loss functions, and excelling particularly on less frequent labels.
This paper proposes a new deep learning architecture for context-based multi-label multi-task emotion recognition. The architecture is built from three main modules: (1) a body features extraction module, which is a pre-trained Xception network, (2) a scene features extraction module, based on a modified VGG16 network, and (3) a fusion-decision module. Moreover, three categorical and three continuous loss functions are compared in order to point out the importance of the synergy between loss functions when it comes to multi-task learning. Then, we propose a new loss function, the multi-label focal loss (MFL), based on the focal loss to deal with imbalanced data. Experimental results on EMOTIC dataset show that MFL with the Huber loss gave better results than any other combination and outperformed the current state of art on the less frequent labels.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available