期刊
BIOMEDICAL SIGNAL PROCESSING AND CONTROL
卷 89, 期 -, 页码 -出版社
ELSEVIER SCI LTD
DOI: 10.1016/j.bspc.2023.105756
关键词
Multimodality; MTFN-SAM network; Classroom fatigue; Feature fusion
This paper proposes a multimodal tensor fusion network based on the self-attention mechanism for classroom fatigue recognition. The experimental results demonstrate that the network has better performance in multimodal feature fusion.
This paper presents a novel multimodal tensor fusion network based on the self-attention mechanism (MTFN-SAM) for classroom fatigue recognition. The LSTM network is first employed to learn the fatigue information of different modalities over time. Then, a self-attention mechanism is utilized to capture the connections within the modal features. Finally, a multimodal tensor fusion layer is employed to fuse features from multiple modalities, including electroencephalogram (EEG), heart rate (HR), galvanic skin response (GSR), and acceleration (ACC). The classification accuracy of the experimental results was compared with TFN, LWF, LSTM, and other fusion algorithms, confirming that the proposed MTFN-SAM network had better performance in multimodal feature fusion. Additionally, we found that as the number of modalities increased, the information contained in the multimodal fusion became more abundant, as confirmed by the experimental results (94.11% accuracy after EEG+HR+GSR+ACC fusion). Thus, the method could efficiently identify students' classroom fatigue.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据