Multimodal information improves the rapid detection of mental fatigue

One of the major challenge in the detection of mental states is improving the accuracy of brain activity-based detectors with additional information from extracranial signals. We assessed the suitability, for real-time mental fatigue detection, of EEG, EOG and ECG measurements, taken separately or together. Thirteen subjects performed six blocks of switching tasks. For each participant, the block with the lowest error rate from the first two blocks and the block with the highest error rate from the last three blocks were discriminated with a machine learning algorithm (support vector machine). The classification scores obtained with ECG or EOG were greater than would be expected by chance (>50%) for time windows of at least 8s. EEG was the best single mode of detection, with classification scores ranging from 80 +/- 3% with a 4s time window to 94 +/- 2% with a 30s time window. The addition of ECG and EOG features to EEG features significantly increased classification scores for short time windows (e.g., to 86 +/- 3% with a 4s time window, p <0.001). For short time windows (up to 12 s), ECG significantly increased the discriminatory power of EEG, whereas EOG did not. These results demonstrate that mental state detection on the basis of extracerebral measurements is feasible and that a combination of EEG and ECG is particularly appropriate for the rapid detection of mental fatigue. (C) 2013 Elsevier Ltd. All rights reserved.