期刊
APPLIED INTELLIGENCE
卷 51, 期 10, 页码 6609-6621出版社
SPRINGER
DOI: 10.1007/s10489-020-02139-8
关键词
Engagement prediction; Spatiotemporal network; Facial spatial and temporal information; LSTM network with global attention
资金
- Key Realm R and D Program of Guangzhou [202007030005]
- Guangdong Natural Science Foundation [2019A1515011375]
- National Natural Science Foundation of China [62076103]
- Scientific Research Foundation of Graduate School of South China Normal University [2019LKXM031]
- Special Funds for the Cultivation of Guangdong College Students' Scientific and Technological Innovation [pdjh2020a0145]
In this paper, a novel model called DFSTN was presented for engagement prediction, achieving good results on the DAiSEE dataset and outperforming many existing works. The model combines SENet for extracting facial spatial features and LSTM with GALN for generating an attentional hidden state.
Recently, online learning has been gradually accepted and approbated by the public. In this context, an effective prediction of students' engagement can help teachers obtain timely feedback and make adaptive adjustments to meet learners' needs. In this paper, we present a novel model called the Deep Facial Spatiotemporal Network (DFSTN) for engagement prediction. The model contains two modules: the pretrained SE-ResNet-50 (SENet), which is used for extracting facial spatial features, and the Long Short Term Memory (LSTM) Network with Global Attention (GALN), which is employed to generate an attentional hidden state. The training strategy of the model is different with changes of the performance metric. The DFSTN can capture facial spatial and temporal information, which is helpful for sensing the fine-grained engaged state and improving the engagement prediction performance. We evaluate the methods on the Dataset for Affective States in E-Environments (DAiSEE) and obtain an accuracy of 58.84% in four-class classification and a Mean Square Error (MSE) of 0.0422. The results show that our method outperforms many existing works in engagement prediction on DAiSEE. Additionally, the robustness of our method is also exhibited by experiments on the EmotiW-EP dataset.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据