Robust Hand Gesture Recognition Based on RGB-D Data for Natural Human-Computer Interaction

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture. Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm. The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD). On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures. Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures. Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.

Automated summary

This paper presents a robust RGB-D data-based recognition method for static and dynamic hand gestures, utilizing algorithms like Distance Transform and K-Curvature-Convex Defects Detection for gesture identification and feature vector construction, and proposing recognition algorithms. Additionally, a unifying feature descriptor is generated for dynamic gestures by combining Euclidean distance and skeleton feature ratios for recognition. Extensive experiments validate the real-time application of the gesture recognition algorithm.