A novel apple fruit detection and counting methodology based on deep learning and trunk tracking in modern orchard

Accurate count of fruits is important for producers to make adequate decisions in production management. Although some algorithms based on machine vision have been developed to count fruits which were all implemented by tracking fruits themselves, those algorithms often make mismatches or even lose targets during the tracking process due to the large number of highly similar fruits in appearance. This study aims to develop an automated video processing method for improving the counting accuracy of apple fruits in orchard environment with modern vertical fruiting-wall architecture. As the trunk is normally larger than fruits and appears clearly in the video, the trunk is thus selected as a single-object tracking target to reach a higher accuracy and higher speed tracking than the commonly used method of fruit-based multi-object tracking. This method was trained using a YOLOv4-tiny network integrated with a CSR-DCF (channel spatial reliability-discriminative correlation filter) algorithm. Reference displacement between consecutive frames was calculated according to the frame motion trajectory for predicting possible fruit locations in terms of previously detected positions. The minimum Euclidean distance of detected fruit position and the predicted fruit position was calculated to match the same fruits between consecutive video frames. Finally, a unique ID was assigned to each fruit for counting. Results showed that mean average precision of 99.35% for fruit and trunk detection was achieved in this study, which could provide a good basis for fruit accurate counting. A counting accuracy of 91.49% and a correlation coefficient R-2 of 0.9875 with counting performed by manual counting were reached in orchard videos. Besides, proposed counting method can be implemented on CPU at 2 ~ 5 frames per second (fps). These promising results demonstrate the potential of this method to provide yield data for apple fruits or even other types of fruits.

Automated summary

This study develops an automated video processing method to improve the counting accuracy of apple fruits in orchard environment. By tracking the trunk instead of the fruits, a higher accuracy and speed can be achieved. The method combines YOLOv4-tiny network and CSR-DCF algorithm for training and utilizes frame displacement to predict fruit locations. Results show a high counting accuracy in orchard videos.