Real-time kiwifruit detection in orchard using deep learning on Android™ smartphones for yield estimation

Fast and accurate detection of kiwifruit in orchard under natural environment is the primary technology for yield estimation. Deep learning has become a prevalent way of fruit detection and achieved outstanding results. Besides, easy-carry smartphones are getting popular and powerful. In this paper, single shot multibox detector (SSD) with two lightweight backbones MobileNetV2 and InceptionV3 were employed to develop an Android APP named KiwiDetector for field kiwifruit detection. An 8-bit quantization method was used to compress model size and improve detection speed by quantizing weight tensor and activation function data of convolutional neural networks from 32-bit floating point to 8-bit integer. Detection test was performed on 100 selected kiwifruit field images with resolution of 3,968 x 2,976 pixels using the four models on a HUAWEI P20 smartphone. Results showed that MobileNetV2, quantized MobileNetV2, InceptionV3, and quantized InceptionV3 obtained true detected rate (TDR) of 90.8%, 89.7%, 87.6%, and 72.8%, respectively. The TDR of MobileNetV2 and quantized MobileNetV2 was generally consistent and higher than InceptionV3 and quantized InceptionV3. For processing an image on the smartphone, MobileNetV2, quantized MobileNetV2, InceptionV3, and quantized InceptionV3 took about 163 ms, 103 ms, 1085 ms, and 685 ms with model sizes of 17.5 MB, 4.5 MB, 96.1 MB, and 24.1 MB, respectively. Quantized MobileNetV2 reached a significant TDR with the fastest detection speed and the smallest model size. The results indicated that the proposed Android APP is promising for yield estimation.