Computer vision based detection of external defects on tomatoes using deep learning

Sorting machines use computer vision (CV) to separate food items based on various attributes. For instance, sorting based on size and colour are commonly used in commercial machines. However, detecting external defects using CV remains an open problem. This paper presents an experimental contribution to external defect detection using deep learning. An uncensored dataset with 43,843 images including external defects was built during this study. The dataset is heavily imbalanced towards the healthy class, and it is available online. Deep residual neural network (ResNet) classifiers were trained that are capable of detecting external defects using feature extraction and fine-tuning. The results show that fine-tuning outperformed feature extraction, revealing the benefit of training additional layers when sufficient data samples are available. The best model was a ResNet50 with all its layers fine-tuned. This model achieved an average precision of 94.6% on the test set. The optimal classifier had a recall of 86.6% while maintaining a precision of 91.7%. The posterior class-conditional distributions of the classifier scores showed that the key to classifier success lies in its almost ideal healthy class distribution. The results also explain why the model does not confuse stems/calyxes with external defects. The best model constitutes a milestone for detecting external defects using CV. Because deep learning does not require feature engineering or prior knowledge about the dataset content, the methodology may also work well with other foods. (C) 2019 IAgrE. Published by Elsevier Ltd. All rights reserved.