Quantifying Emotional Similarity in Speech

This study proposes the novel formulation of measuring emotional similarity between speech recordings. This formulation explores the ordinal nature of emotions by comparing emotional similarities instead of predicting an emotional attribute, or recognizing an emotional category. The proposed task determines which of two alternative samples has the most similar emotional content to the emotion of a given anchor. This task raises some interesting questions. Which is the emotional descriptor that provide the most suitable space to assess emotional similarities? Can deep neural networks (DNNs) learn representations to robustly quantify emotional similarities? We address these questions by exploring alternative emotional spaces created with attribute-based descriptors and categorical emotions. We create the representation using a DNN trained with the triplet loss function, which relies on triplets formed with an anchor, a positive example, and a negative example. We select a positive sample that has similar emotion content to the anchor, and a negative sample that has dissimilar emotion to the anchor. The task of our DNN is to identify the positive sample. The experimental evaluations demonstrate that we can learn a meaningful embedding to assess emotional similarities, achieving higher performance than human evaluators asked to complete the same task.

Automated summary

This study proposes a new formula for measuring emotional similarity between speech recordings. Instead of predicting emotional attributes or recognizing emotional categories, this formulation explores the ordinal nature of emotions by comparing emotional similarities. The study addresses questions about which emotional descriptors provide the most suitable space to assess emotional similarities and whether deep neural networks can learn representations to quantify emotional similarities robustly. By creating alternative emotional spaces using attribute-based descriptors and categorical emotions, the study shows that a meaningful embedding can be learned to assess emotional similarities, outperforming human evaluators in the same task.