Faster R-CNN for multi-class fruit detection using a robotic vision system

An accurate and real-time image based multi-class fruit detection system is important for facilitating higher level smart farm tasks such as yield mapping and robotic harvesting. Robotic harvesting can reduce the costs of labour and increase fruit quality. This paper proposes a deep learning framework for multi-class fruits detection based on improved Faster R-CNN. The proposed framework includes fruits image library creation, data argumentation, improved Faster RCNN model generation, and performance evaluation. This work is a pioneer to create a multi-labeled and knowledge-based outdoor orchard image library using 4000 images in the real world. Also, improvement of the convolutional and pooling layers is achieved to have a more accurate and faster detection. The test results show the proposed algorithm has achieved higher detecting accuracy and lower processing time than the traditional detectors, which has excellent potential to build an autonomous and real-time harvesting or yield mapping/estimation system. (C) 2019 Elsevier B.V. All rights reserved.