A multimodal approach to estimating vigilance in SSVEP-based BCI

Brain-computer interface (BCI) is a communication system that allows a direct connection between the human brain and external devices, which is able to provide assistance and improve the quality of life for people with disabilities. Vigilance is an important cognitive state and plays an important role in human-computer interac-tion. In BCI tasks, the low-vigilance state of the BCI user would lead to the performance degradation. Therefore, it is desirable to develop an efficient method to estimate the vigilance state of BCI users. In this study, we built a 4 -target BCI system based on steady-state visual evoked potential (SSVEP) for cursor control. Electroencephalo-gram (EEG) and electrooculogram (EOG) were recorded simultaneously from 18 subjects during a 90-min continuous cursor-control BCI task. We proposed a multimodal vigilance estimating network, named MVENet, to estimate the vigilance state of BCI users through the multimodal signals. In this architecture, a spatial -temporal convolution module with an attention mechanism was adopted to explore the temporal-spatial infor-mation of the EEG features, and a long short-term memory module was utilized to learn the temporal de-pendencies of EOG features. Moreover, a fusion mechanism was built to fuse the EEG representations and EOG representations effectively. Experimental results showed that the proposed network achieved a better perfor-mance than the compared methods. These results demonstrate the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users.

Automated summary

In this study, a 4-target BCI system based on SSVEP was built to assist people with disabilities. EEG and EOG data were recorded from 18 subjects during a 90-min continuous task, and a multimodal vigilance estimating network, MVENet, was proposed to estimate the vigilance state of BCI users. Experimental results showed that the network achieved better performance than the compared methods, demonstrating the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users.