Insect recognition based on complementary features from multiple views

Insect pest recognition has always been a significant branch of agriculture and ecology. The slight variance among different kinds of insects in appearance makes it hard for human experts to recognize. It is increasingly imperative to finely recognize specific insects by employing machine learning methods. In this study, we proposed a feature fusion network to synthesize feature presentations in different backbone models. Firstly, we employed one CNN-based backbone ResNet, and two attention-based backbones Vision Transformer and Swin Transformer to localize the important regions of insect images with Grad-CAM. During this process, we designed new architectures for these two Transformers to enable Grad-CAM to be applicable in such attention-based models. Then we further proposed an attention-selection mechanism to reconstruct the attention area by delicately integrating the important regions, enabling these partial but key expressions to complement each other. We only need part of the image scope that represents the most crucial decision-making information for insect recognition. We randomly selected 20 species of insects from the IP102 dataset and then adopted all 102 kinds of insects to test the classification performance. Experimental results show that the proposed approach outperforms other advanced CNN-based models. More importantly, our attention-selection mechanism demonstrates good robustness to augmented images.

Automated summary

Insect pest recognition is an important field in agriculture and ecology, but the slight variations in appearance among different insect species make it difficult for human experts to recognize them. Therefore, the use of machine learning methods for insect recognition is becoming increasingly important. In this study, we proposed a feature fusion network that combines feature representations from different backbone models. We used CNN-based ResNet, attention-based Vision Transformer, and Swin Transformer backbones to localize important insect image regions. We also developed an attention-selection mechanism that integrates these important regions to reconstruct attention areas and improve insect recognition.