Cooperative Sensing and Wearable Computing for Sequential Hand Gesture Recognition

Hand gestures recognition (HGR) has been considered as one of the crucial research fields of human-computer interaction (HCI). Computer vision is a very active research field in the HGR, traditional vision-based methods, which used camera and ultrasonic/optical sensor to collect the videos or images of the hand gestures shown by participants, have some limitations, such as fixed in-lab location, complex lighting conditions, and cluttered backgrounds. In order to provide new approaches, we described the development of a novel hand gesture recognition system that combined wearable armband and smart glove made by customizable pressure sensor arrays to detect sequential hand gestures. A deep learning technique long short-term memory (LSTM) algorithm had been computed to build an effective model to classify hand gestures by training and testing the collected inertial measurement unit (IMU), electromyographic (EMG), and finger and palm's pressure data. Furthermore, we built a relatively large database of ten sequential hand gestures consisted by five dynamic gestures and five air gestures collected from ten participants. Our experimental results showed an outstanding classification performance of the proposed LSTM algorithm. These findings have promising implications for sequential hand gesture recognition and the HCI research status.