Hand pose estimation based on regression method from monocular RGB cameras for handling occlusion

Hand pose estimation is a significant research topic for various computer vision applications. Nonetheless, reliable and robust pose estimation with existing methods remains challenging due to the complex anatomy of the hand and the varying shapes and sizes of hands. The traditional approach involved using depth sensors or multi-camera setups. However, with the advent of deep learning, there has been a shift towards using deep neural networks to learn, grasp, and manipulate objects accurately. In this paper, we propose an end-to-end framework called ResUnet network that can efficiently detect and estimate the position of a human hand from a monocular RGB image. Our proposal aims to handle occlusion issue during the hand-object interaction in real-time. The ResUnet architecture includes three modules, feature extraction, 2D pose regression, and 3D hand estimation. The first module extracts the feature maps of the cropped hand to generate 2D heatmaps. The second module uses the previous outputs to regress the 2D pose coordinates employing Latent Heatmaps Representation (LHR). The last module concatenates the intermediate features with the upsampling block to process 3D regression and predict the 3D bones using a tree structure of the hand. Quantitative and qualitative results on three datasets GANerated, SynthHands, and Stereo Hand Pose Tracking Benchmark (STB), consistently demonstrate that our regression approach outperforms the current state-of-the-art hand pose estimation methods.

Automated summary

Hand pose estimation is a significant research topic in computer vision applications. This paper proposes an end-to-end framework called ResUnet network that can efficiently detect and estimate the position of a human hand from a monocular RGB image. The quantitative and qualitative results demonstrate that our regression approach outperforms the current state-of-the-art hand pose estimation methods on three datasets.