In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

This paper proposes a method to estimate the 6D pose of an object grasped by a robot hand using RGB cameras mounted on the palm and visuotactile sensors installed at the fingertips. It can deal with objects made from a wide range of materials thanks to combining the two types of sensors. The method allows a robot to robot to perform in-hand pose estimation while holding the object, eliminating the need for preparatory actions or particular environmental backgrounds. The mechanism at the back of the method includes deep-learning-based background subtraction and denoising auto-encoder-based sensor fusion. With the poses estimated using the proposed method, a robot controller can rectify the grasping uncertainty and adjust the robot motion to move an object toward required goals with satisfying accuracy. We conduct various studies and analyses in the experimental Section to understand the proposed method's advantages and disadvantages. The results demonstrate the benefits of the proposed combination and mechanism. They also provide essential knowledge to readers considering using a similar configuration for estimating object poses.

Automated summary

This paper presents a method for estimating the 6D pose of an object grasped by a robot hand using RGB cameras on the palm and visuotactile sensors on the fingertips. By combining the two types of sensors, it can handle objects made from various materials. The method allows in-hand pose estimation without the need for preparation or specific environmental backgrounds. The proposed method includes deep-learning-based background subtraction and denoising auto-encoder-based sensor fusion. The results demonstrate the benefits of the proposed combination and mechanism, providing essential knowledge for readers considering similar configurations for pose estimation.