Segmentation and recognition of filed sweet pepper based on improved self-attention convolutional neural networks

Automatic and accurate recognition of the parts to be picked is the key to realize the intelligent picking of sweet pepper. However, pepper fruits are always covered by other organs, and small objects such as stems and shoots are difficult to be recognized by machines or cameras under certain extreme conditions. To accurately segment and recognize all kinds of objects in sweet pepper images captured at night, three experiments were performed in this paper, and an enhanced model based on convolutional neural networks was eventually achieved. In experiment I, several semantic segmentation networks were trained on a small data set, and the full-resolution residual network (FRRN) was taken as a primary network. Then, the impact of resolution of input images on the segmentation effect was investigated in experiment II. To strengthen the feature presentation of inconspicuous objects, the position attention module was appended on top of the FRRN in experiment III. This architecture was further trained to provide more precise segmentation results. The experimental result shows that the mean intersection over union is 78.88%, which is at least 1.94% points higher than other models, and the mean pixel accuracy is 97.94% on the test set. The proposed method has higher generalization performance when facing unforeseen picking situations; meanwhile, it is generic and can be applied to other fruits and vegetables.

Automated summary

This paper proposes an improved model based on convolutional neural networks for accurate segmentation and recognition of various objects in sweet pepper images captured at night. The experimental results show that the proposed method achieves higher segmentation performance compared to other models and has good generalization performance when facing unforeseen picking situations.