Dimensional Affect Uncertainty Modelling for Apparent Personality Recognition

Despite achieving impressive performance, dimensional affect or emotion recognition from faces is largely based on uncertainty-unaware models that predict only point estimates. Modelling uncertainty is important to learn reliable facial emotion recognition models with the abilities to (a). holistically quantify predictive uncertainty estimates and (b). propagate those estimates to the benefit of downstream behavioural analysis tasks. In this work, we first quantify uncertainties in dimensional emotion recognition by adopting the framework of epistemic (model) and aleatoric (data) uncertainty categorisation. Then for evaluating the practical utility of uncertainty-aware emotion predictions, we introduce them in learning an important downstream task, apparent personality recognition. To this end, we ask two questions: how to effectively (a). use already known behavioural attributes (emotions) in a downstream task (personality recognition) and (b). summarise global temporal context from uncertainty-aware emotion predictions fused with image embeddings. Answering these questions, we learn a conditional latent variable model building on recently proposed neural latent variable models. Our experiments on two in-the-wild datasets, SEWA for emotion recognition and ChaLearn for personality recognition, demonstrate that fusion of epistemic and aleatoric emotion uncertainties significantly improves personality recognition performance, with similar to 42% relative improvement in Pearson correlation coefficient, leading to a new state-of-the-art.

Automated summary

This study focuses on the uncertainty issue in facial emotion recognition and applies uncertainty to personality recognition tasks. Experimental results demonstrate that the fusion of uncertainty greatly improves the performance of personality recognition.