Real emotion seeker: recalibrating annotation for facial expression recognition

Facial expression recognition (FER) is a challenging classification task. Due to the subjectivity and ambiguity of performers and spectators, compound facial expression is hard to be represented by one-hot label. In this paper, a simple but efficient method, named real emotion seeker (RES), is proposed to recalibrate the annotation of sample to latent expression distribution besides one-hot label. In particular, subjective implicit knowledge is transformed into posterior distribution which is specific to each FER data set through Bayesian inference, thus enhancing universality and authenticity. The posterior distribution is then combined with one-hot label to form the recalibrated annotation as an additional supervision, guiding the prediction more realistic. Our proposed method is independent of the backbone network and can improve the accuracy significantly by an average of 3.16% with no burden for training and inference. Extensive experiments show that RES can obtain consistent prediction with human subjective intuition. Results on three in-the-wild data sets demonstrate that our approach achieves advanced results with 90.38% on RAF-DB, 90.34% on FERPlus and 62.63% on AffectNet.

Automated summary

This paper proposes a method called real emotion seeker (RES) to recalibrate compound facial expressions by incorporating subjective implicit knowledge through Bayesian inference and posterior distribution. The recalibrated annotation, combined with one-hot label, guides more realistic prediction and significantly improves accuracy in facial expression recognition.