Towards Recognizing Emotion in the Latent Space

Emotion recognition is usually achieved by collecting features (physiological signals, events, facial expressions, etc.) to predict an emotional ground truth. This ground truth is arguably unreliable due to its subjective nature. In this paper, we introduce a new approach to measure the magnitude of an emotion in the latent space of a Neural Network without the need for a subjective ground truth. Our data consists of physiological measurements during video gameplay, game events, and subjective rankings of game events for the validation of our model. Our model encodes physiological features into a latent variable which is then decoded into video game events. We show that the events are ranked in the latent space similarly to the participants' subjective ranks. For instance, our model's ranking is correlated (Kendall tau of 0.91) with the predictability rankings.

Automated summary

This paper introduces a new method to measure the magnitude of an emotion in the latent space of a Neural Network without the need for a subjective ground truth. By processing video game data, the model demonstrates results that are correlated with the subjective rankings of participants.