Affective brain-computer interfaces: Choosing a meaningful performance measuring metric

Affective brain-computer interfaces are a relatively new area of research in affective computing. Estimation of affective states can improve human-computer interaction as well as improve the care of people with severe disabilities. To assess the effectiveness of EEG recordings for recognizing affective states, we used data collected in our lab as well as the publicly available DEAP database. We also reviewed the articles that used the DEAP database and found that a significant number of articles did not consider the presence of the class imbalance in the DEAP. Failing to consider class imbalance creates misleading results. Further, ignoring class imbalance makes the comparison of the results between studies using different datasets impossible, since different datasets will have different class imbalances. Class imbalance also shifts the chance level, hence it is vital to consider class bias while determining if the results are above chance. To properly account for the effect of class imbalance, we suggest the use of balanced accuracy as a performance metric, and its posterior distribution for computing credible intervals. For classification, we used features from the literature as well as theta beta-1 ratio. Results from DEAP and our data suggest that the beta band power, theta band power, and theta beta-1 ratio are better feature sets for classifying valence, arousal, and dominance, respectively.