Classification Accuracy from the Perspective of the User: Real-Time Interaction with Physiological Computing

The accurate classification of psychophysiological data is an important determinant of the quality when interacting with a physiological computing system. Previous research has focused on classification accuracy of psychophysiological data in purely mathematical terms but little is known about how accuracy metrics relate to users' perceptions of accuracy during real-time interaction. A group of 14 participants watched a series of movie trailers and were asked to subjectively indicate their level of interest in a binary high/low fashion. Psychophysiological data (EEG, ECG and SCL) were used to create a binary classification of interest via a Support Vector Machine (SVM) algorithm. After a period of training, participants received real-time feedback from the classification algorithm and perceptions of accuracy were assessed. The purpose of the study was to compare mathematical classification accuracy with the perceived accuracy of the system as experienced by the users. Results indicated that perceived accuracy was subject to a number of psychological biases resulting from expectations, entrainment and development of trust. The F1 score was generally a significant predictor of perceived accuracy.