期刊
EXPERT SYSTEMS
卷 38, 期 5, 页码 -出版社
WILEY
DOI: 10.1111/exsy.12490
关键词
hand gesture recognition; multimodal data fusion; parallel CNN; sEMG signal
资金
- CAS Interdisciplinary Innovation Team [JCTD-2018-11]
- National Key R&D Program of China [2018YFB1304600]
- Natural Science Foundation of China [51575412, 51775541]
- EU Seventh Framework Programme (FP7)-ICT [611391]
In this paper, a method based on multimodal data fusion and multiscale parallel convolutional neural network is proposed to improve the accuracy and reliability of hand gesture recognition. Experiments on a self-made database verified the effectiveness and superiority of the method, which was also successfully applied to a seven-degree-of-freedom bionic manipulator for robotic manipulation using hand gestures.
Hand gesture recognition plays an important role in human-robot interaction. The accuracy and reliability of hand gesture recognition are the keys to gesture-based human-robot interaction tasks. To solve this problem, a method based on multimodal data fusion and multiscale parallel convolutional neural network (CNN) is proposed in this paper to improve the accuracy and reliability of hand gesture recognition. First of all, data fusion is conducted on the sEMG signal, the RGB image, and the depth image of hand gestures. Then, the fused images are generated to two different scale images by downsampling, which are respectively input into two subnetworks of the parallel CNN to obtain two hand gesture recognition results. After that, hand gesture recognition results of the parallel CNN are combined to obtain the final hand gesture recognition result. Finally, experiments are carried out on a self-made database containing 10 common hand gestures, which verify the effectiveness and superiority of the proposed method for hand gesture recognition. In addition, the proposed method is applied to a seven-degree-of-freedom bionic manipulator to achieve robotic manipulation with hand gestures.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据