weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming

Selective weed treatment is a critical step in autonomous crop management as related to crop health and yield. However, a key challenge is reliable and accurate weed detection to minimize damage to surrounding plants. In this letter, we present an approach for dense semantic weed classification with multispectral images collected by a micro aerial vehicle (MAV). We use the recently developed encoder-decoder cascaded convolutional neural network, SegNet, that infers dense semantic classes while allowing any number of input image channels and class balancing with our sugar beet and weed datasets. To obtain training datasets, we established an experimental field with varying herbicide levels resulting in field plots containing only either crop or weed, enabling us to use the normalized difference vegetation index as a distinguishable feature for automatic ground truth generation. We train six models with different numbers of input channels and condition (fine tune) it to achieve similar to 0.8 F1-score and 0.78 area under the curve classification metrics. For the model deployment, an embedded Graphics Processing Unit (GPU) system (Jetson TX2) is tested for MAV integration. Dataset used in this letter is released to support the community and future work.