KeypointMask: A Real-Time Instance Segmentation for Oblique Object Detection in Robotic Picking

In vision-based robotic grasping, instance segmentation locates the object in the 2D image, and cropping the 2D mask area points cloud to estimate the object pose. The latest deep learning algorithm has a corner case and is ignored by the current state-of-the-art instance segmentation methods. In the eye-to-hand robotic picking scenario, the 3D camera is mounted on the top of the robotic arm and captures the color and depth image in the bird's eye view manner. In this situation, the introclass suppression between the same object causes a serious missing detection issue for diagonally arranged or stacked objects with axis-aligned bounding boxes. This paper proposes a real-time occlusion and oblique-friendly instance segmentation framework, Keypoint-Mask, to solve the above problem. We empirically demonstrate the effectiveness of our framework on the MS COCO and industrial-level datasets. Our framework achieved 35 FPS and 37.6% AP(mask) on the COCO test-dev instance segmentation, outperforming similar lightweight models such as Blender-Mask, Center-Mask, and Yolact. Furthermore, Keypoint-Mask achieved the highest 71.7% AP(mask) on our collected case dataset, filling the gap that all the state-of-the-art models cannot reach. The dataset is labeled in the COCO format and made publicly available.

Automated summary

This paper proposes a real-time occlusion and oblique-friendly instance segmentation framework called Keypoint-Mask to solve the missing detection issue in robotic grasping. The framework achieved excellent results in multiple experiments and outperformed other similar models.