A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion With FMCW Radar

For hand gesture recognition (HGR) sensing based on frequency-modulated continuous wave (FMCW) radar, it is challenging to cope with the ambiguity and variability of gesture motions performed by different peoples due to the private habits or position, which inevitably reduce the robustness of HGR sensing. To solve this problem, we propose a novel dual-flow fusion deformable residual network (DFDRN). More detailed gesture information, time frequency maps and spectrum map videos, are fed into DFDRN, which improves the generalization and motion modeling ability of HGR through the extensive feature space. To further improve the robustness of HGR sensing, the deformable block is exploited to replace the convolution. To focus on the important features of dual-flow fusion, the convolutional block attention module (CBAM) is embedded in the backbone of DFDRN. Experimental results show that the accuracy of DFDRN, which can reach 97.5%, is obviously better than the state-of-the-art HGR methods.

Automated summary

A novel dual-flow fusion deformable residual network (DFDRN) is proposed in this study to improve the generalization and motion modeling ability of hand gesture recognition (HGR) by utilizing time frequency maps and spectrum map videos, as well as enhancing robustness through deformable blocks and convolutional block attention modules. Experimental results show that DFDRN achieves an accuracy of 97.5%, significantly outperforming state-of-the-art HGR methods.