Journal
IEEE ROBOTICS AND AUTOMATION LETTERS
Volume 7, Issue 4, Pages 10272-10279Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LRA.2022.3193251
Keywords
Deep learning; gesture recognition; human-robot interaction
Categories
Funding
- Spanish State Research Agency through the ROCOTRANSP Project [PID2019-106702RBC21/AEI/10.13039/501100011033]
- EU [H2020-ICT-2020-2-101016906]
Ask authors/readers for more resources
In this paper, an efficient and reliable deep-learning approach for hand gesture recognition in robot communication is presented. The approach utilizes visual information to extract hand landmarks and predict gestures, achieving high accuracy and real-time performance.
In this paper, we present an efficient and reliable deep-learning approach that allows users to communicate with robots via hand gesture recognition. Contrary to other works which use external devices such as gloves [1] or joysticks [2] to tele-operate robots, the proposed approach uses only visual information to recognize user's instructions that are encoded in a set of pre-defined hand gestures. Particularly, the method consists of two modules which work sequentially to extract 2D landmarks of hands -ie. joints positions- and to predict the hand gesture based on a temporal representation of them. The approach has been validated in a recent state-of-the-art dataset where it outperformed other methods that use multiple pre-processing steps such as optical flow and semantic segmentation. Our method achieves an accuracy of 87.5% and runs at 10 frames per second. Finally, we conducted real-life experiments with our IVO robot to validate the framework during the interaction process.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available