Journal
SENSORS
Volume 22, Issue 7, Pages -Publisher
MDPI
DOI: 10.3390/s22072513
Keywords
human-UAV interaction; hybrid-based hand gesture recognition; hand-gesture-based recognition; IMU-based motion capture system; deep learning
Funding
- Unmanned Vehicles Core Technology Research and Development Program through the National Research Foundation of Korea (NRF)
- Unmanned Vehicle Advanced Research Center (UVARC) - Ministry of Science and ICT (MSIT), the Republic of Korea [2020M3C1C1A01084900]
- the Competency Development Program for Industry Specialists of Korean Ministry of Trade, Industry and Energy (MOTIE) [N0002428]
- NRF - Korea Government (MSIT) [2020R1C1C1007739]
- National Research Foundation of Korea [2020R1C1C1007739, 2020M3C1C1A01084900] Funding Source: Korea Institute of Science & Technology Information (KISTI), National Science & Technology Information Service (NTIS)
Ask authors/readers for more resources
In this paper, a hybrid hand gesture system combining an IMU-based motion capture system and a vision-based gesture system is proposed to improve real-time performance. The system utilizes real-time object detection and an estimated orientation sensed by a thumb-mounted micro-IMU to allow flexible drone operation.
As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of dynamic gesture because it is difficult to estimate the pose of multi-dimensional hand gestures in 2D images. This leads to complex algorithms, including tracking in addition to detection, to recognize dynamic gestures, but they are not suitable for human-UAV interaction (HUI) systems that require safe design with high real-time performance. Therefore, in this paper, we propose a hybrid hand gesture system that combines an inertial measurement unit (IMU)-based motion capture system and a vision-based gesture system to increase real-time performance. First, IMU-based commands and vision-based commands are divided according to whether drone operation commands are continuously input. Second, IMU-based control commands are intuitively mapped to allow the UAV to move in the same direction by utilizing estimated orientation sensed by a thumb-mounted micro-IMU, and vision-based control commands are mapped with hand's appearance through real-time object detection. The proposed system is verified in a simulation environment through efficiency evaluation with dynamic gestures of the existing vision-based system in addition to usability comparison with traditional joystick controller conducted for applicants with no experience in manipulation. As a result, it proves that it is a safer and more intuitive HUI design with a 0.089 ms processing speed and average lap time that takes about 19 s less than the joystick controller. In other words, it shows that it is viable as an alternative to existing HUI.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available