GKNet: Grasp keypoint network for grasp candidates detection

Contemporary grasp detection approaches employ deep learning to achieve robustness to sensor and object model uncertainty. The two dominant approaches design either grasp-quality scoring or anchor-based grasp recognition networks. This paper presents a different approach to grasp detection by treating it as keypoint detection in image-space. The deep network detects each grasp candidate as a pair of keypoints, convertible to the grasp representation g = {x,y,w,theta}( T ), rather than a triplet or quartet of corner points. Decreasing the detection difficulty by grouping keypoints into pairs boosts performance. To promote capturing dependencies between keypoints, a non-local module is incorporated into the network design. A final filtering strategy based on discrete and continuous orientation prediction removes false correspondences and further improves grasp detection performance. GKNet, the approach presented here, achieves a good balance between accuracy and speed on the Cornell and the abridged Jacquard datasets (96.9% and 98.39% at 41.67 and 23.26 fps). Follow-up experiments on a manipulator evaluate GKNet using four types of grasping experiments reflecting different nuisance sources: static grasping, dynamic grasping, grasping at varied camera angles, and bin picking. GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and moderate clutter. The results confirm the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robustness to expected nuisance factors.

Automated summary

Contemporary grasp detection approaches often rely on deep learning to handle uncertainties in sensors and object models. This paper presents a novel approach to grasp detection by treating it as keypoint detection in image-space. The proposed method detects grasp candidates as pairs of keypoints, and incorporates a non-local module to capture dependencies between keypoints. Experimental results show that the approach achieves a good balance between accuracy and speed, and demonstrates robustness to various nuisance factors in different types of grasping experiments.