Human-Robot Interaction (HRI) through hand gestures for possible future war robots: A leap motion controller application

In this article, the futuristically possible human (commander)-robot (soldier) interaction (HRI) based on effective hand gesture recognition is discussed. As methodologically, Leap Motion Controller (LMC), which is frequently used in virtual reality applications, was used to obtain hand gesture features. Only the relevant distance of the fingers to each other and to the normal of the hand is considered as a feature and high performance is questioned under these constraints. Then performances of six hand gesture recognition methods, classified as light, medium weight, and complex, were examined with random dynamic movements and in different frame numbers. The performance of the proposed cumulative distribution function (CDF) based deep neural network (DNN) approach has achieved an accuracy of 88.44%. With this result, an improvement of 4.76% has been achieved compared to the second closest method, Kullback Leibler Divergence, by using the proposed method. Although limited features, high performance has been achieved. There is no mechanical or electronic robot design in the study; however, the computer used as the decision mechanism of the robot was modeled and made ready for application. In this sense, we believe wholeheartedly that in the future, this work can be a pioneer study in the military field.

Automated summary

This article discusses the possibilities of future human-robot interaction (HRI) based on effective hand gesture recognition. The Leap Motion Controller (LMC) was used to obtain hand gesture features, and only the relevant distances between fingers and the hand's normal were considered. Six hand gesture recognition methods of different complexities were examined for performance, with the proposed CDF-based DNN approach achieving an accuracy of 88.44%. This marks a 4.76% improvement compared to the second best method, Kullback Leibler Divergence. Although limited in features, high performance was achieved in the study, making it a potential pioneering study in the military field in the future.