Manipulator grabbing position detection with information fusion of color image and depth image using deep learning

In order to ensure stable gripping performance of manipulator in a dynamic environment, a target object grab setting model based on the candidate region suggestion network is established with the multi-target object and the anchor frame generation measurement strategy overcoming external environmental interference factors such as mutual interference between objects and changes in illumination. In which, the success rate of model detection is improved by adding small-scale anchor values for small area grabbing target position detection. Further, 94.3% crawl detection success rate is achieved on the multi-target detection data sets using the information fusion of color image and depth image. The methods in this paper effectively improve the model's robustness and crawl success rate.

Automated summary

A target object grab setting model is established based on the candidate region suggestion network to ensure stable gripping performance of a manipulator in a dynamic environment. The model's detection success rate is improved by adding small-scale anchor values for small area grabbing target position detection, and a 94.3% crawl detection success rate is achieved on multi-target detection data sets through color image and depth image information fusion. These methods effectively enhance the model's robustness and crawl success rate.