Journal
IEEE ROBOTICS AND AUTOMATION LETTERS
Volume 6, Issue 3, Pages 6039-6045Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LRA.2021.3089999
Keywords
Human-Robot interaction; hand gesture recognition; teleoperation; sensor fusion; deep learning
Categories
Funding
- National Natural Science Foundation of China [U1913601, 61625303]
- National Key Research and Development Program of China [2020YFC2007900]
- Opening Project of Shanghai Robot R&D and Transformation Functional Platform
Ask authors/readers for more resources
In this letter, we propose a novel multi-sensor guided hand gesture recognition system for surgical robot teleoperation, using a multi-sensor data fusion model and a multilayer Recurrent Neural Network for multiple hand gestures classification to achieve human-robot collaboration tasks. Results show that the proposed model can achieve higher recognition rate and faster inference speed.
Touch-free guided hand gesture recognition for human-robot interactions plays an increasingly significant role in teleoperated surgical robot systems. Indeed, despite depth cameras provide more practical information for recognition accuracy enhancement, the instability and computational burden of depth data represent a tricky problem. In this letter, we propose a novel multi-sensor guided hand gesture recognition system for surgical robot teleoperation. A multi-sensor data fusion model is designed for performing interference in the presence of occlusions. A multilayer Recurrent Neural Network (RNN) consisting of a Long Short-Term Memory (LSTM) module and a dropout layer (LSTM-RNN) is proposed for multiple hand gestures classification. Detected hand gestures are used to perform a set of human-robot collaboration tasks on a surgical robot platform. Classification performance and prediction time is compared among the LSTM-RNN model and several traditional Machine Learning (ML) algorithms, such as k-Nearest Neighbor (k-NN) and Support Vector Machines (SVM). Results show that the proposed LSTM-RNN classifier is able to achieve a higher recognition rate and faster inference speed. In addition, the present adaptive data fusion system shows a strong anti-interference capability for hand gesture recognition in real-time.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available