Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model

The detection and localization of pineapple fruit must be successfully conducted to realize intelligent picking. This paper proposed a method for detecting and localizing pineapples in natural environments based on binocular stereo vision and an improved YOLO (You Only Look Once) v3 model. In comparison with the original YOLOv3, the improved algorithm did the following two improvement: the DenseNet was added into the Darknet-53 backbone network to optimize the 13 x 13 and 26 x 26 feature layer, and the SPP-net was fused in the 52 x 52 dimension detection module to strengthen the information representation ability of feature map. A binocular camera acquired left and right images. The left image was then input to the improved YOLOv3 model to obtain the position information of pineapples in the image. Stereo matching and parallax calculation of the target pineapple region were completed via the stereo matching algorithm. Finally, the three-dimensional co-ordinates of pineapples were calculated based on the triangulation principle of binocular stereo vision. A series of experiments were run to compare the detection result of this method against YOLOv3, Faster-RCNN (Region Convolutional Neural Network) and Mobilenet-SSD (Single Shot MultiBox Detector) as per their respective F1 score (Balanced Score) and AP (Average Precision) values. On test set with slight occlusion, the F1 score and AP values of the improved YOLOv3 model were 93.18% and 97.55%, respectively. As the occlusion grew severe, the F1 score and AP values decreased to 89.15% and 91.47%, respectively. The improved YOLOv3 model developed in this study had the best detection effect among all models tested. The binocular stereo vision localization experiment showed an average absolute error of 24.414 mm and average relative error of 1.17% at a distance of 1.7-2.7 m. The proposed method may thus be suitable for picking robots detecting and localizing pineapple fruit in natural environments.

Automated summary

This paper proposes a method for detecting and localizing pineapples in natural environments using binocular stereo vision and an improved YOLOv3 model. Experimental results show that the improved model outperforms other models in terms of detection accuracy and localization precision.