Graph-Enhanced Emotion Neural Decoding

Brain signal-based emotion recognition has recently attracted considerable attention since it has powerful potential to be applied in human-computer interaction. To realize the emotional interaction of intelligent systems with humans, researchers have made efforts to decode human emotions from brain imaging data. The majority of current efforts use emotion similarities (e.g., emotion graphs) or brain region similarities (e.g., brain networks) to learn emotion and brain representations. However, the relationships between emotions and brain regions are not explicitly incorporated into the representation learning process. As a result, the learned representations may not be informative enough to benefit specific tasks, e.g., emotion decoding. In this work, we propose a novel idea of graph-enhanced emotion neural decoding, which takes advantage of a bipartite graph structure to integrate the relationships between emotions and brain regions into the neural decoding process, thus helping learn better representations. Theoretical analyses conclude that the suggested emotion-brain bipartite graph inherits and generalizes the conventional emotion graphs and brain networks. Comprehensive experiments on visually evoked emotion datasets demonstrate the effectiveness and superiority of our approach.

Automated summary

Brain signal-based emotion recognition has gained attention for its potential in human-computer interaction. Researchers have attempted to decode human emotions from brain imaging data using emotion and brain representations. However, the relationship between emotions and brain regions is not explicitly incorporated into the representation learning process, leading to insufficient informative representations for specific tasks such as emotion decoding. This work proposes a graph-enhanced emotion neural decoding approach that integrates the relationships between emotions and brain regions into the process, demonstrating its effectiveness and superiority through comprehensive experiments on visually evoked emotion datasets.