Journal
PATTERN RECOGNITION LETTERS
Volume 150, Issue -, Pages 115-121Publisher
ELSEVIER
DOI: 10.1016/j.patrec.2021.07.005
Keywords
Depression recognition; Facial representation; Convolutional neural network; Multimodal learning; Sequential fusion
Categories
Ask authors/readers for more resources
The paper proposes a sequential fusion method for facial depression recognition, which aims to improve depression recognition by exploring the correlation and complementarity between different data modalities in multimodal learning. The results show the superiority of this method against several state-of-the-art alternatives.
In mental health assessment, it is validated that nonverbal cues like facial expressions can be indicative of depressive disorders. Recently, the multimodal fusion of facial appearance and dynamics based on convolutional neural networks has demonstrated encouraging performance in depression analysis. However, correlation and complementarity between different visual modalities have not been well studied in prior methods. In this paper, we propose a sequential fusion method for facial depression recognition. For mining the correlated and complementary depression patterns in multimodal learning, a chained-fusion mechanism is introduced to jointly learn facial appearance and dynamics in a unified framework. We show that such sequential fusion can provide a probabilistic perspective of the model correlation and complementarity between two different data modalities for improved depression recognition. Results on a benchmark dataset show the superiority of our method against several state-of-the-art alternatives. (c) 2021 Elsevier B.V. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available