The Virtual Trackpad: An Electromyography-Based, Wireless, Real-Time, Low-Power, Embedded Hand-Gesture-Recognition System Using an Event-Driven Artificial Neural Network

This brief presents a wireless, low-power embedded system that recognizes hand gestures by decoding surface electromyography (EMG) signals. Ten hand gestures used on commercial trackpads, including pinch, stretch, swipe left, swipe right, scroll up, scroll down, single click, double click, pat, and ok, can be recognized in real time. Features from four differential EMG channels are extracted in multiple time windows. Unlike traditional data segmentation methods, an event-driven method is proposed, with the gesture event detected in the hardware. Feature extraction is triggered only when an event is detected, minimizing computation, memory, and system power. A time-delayed artificial neural network (ANN) is used to predict the gesture from the transient EMG features instead of traditional steady-state features. The ANN is implemented in the microcontroller with a processing time less than 0.2 ms. The detection results are sent wirelessly to a computer. The device weights 15.2 g. A 4.6 g battery supports up to 40 h continuous operation. To our knowledge, this brief shows the first real-time, embedded hand-gesture-recognition system using only transient EMG signals. Experiments with four subjects show that the device can achieve a recognition of ten gestures with an average accuracy of 94%.