Correlation-based structural dropout for convolutional neural networks

Convolutional neural networks (CNNs) easily suffer from the over-fitting problem since they are often over-parameterized in the case of small training datasets. The conventional dropout that drops feature units randomly works well for fully connected networks, but fails to regularize CNNs well due to high spatial correlation of the intermediate features, which allows the dropped information to flow through the network, thus leading to the problem of under-dropping. To better regularize CNNs, some structural dropout methods such as SpatialDropout and DropBlock have been proposed by dropping feature units in continuous regions randomly. However, these methods may suffer from the over-dropping problem by discarding the critical discriminative features, thus limiting the performance of CNNs. To address these issues, we propose a novel structural dropout method, Correlation based Dropout (CorrDrop), to regularize CNNs by dropping feature units based on feature correlation. Unlike the previous dropout methods, our CorrDrop can focus on the discriminative information and drops features in a spatial-wise or channel wise manner. Extensive experiments on different datasets, network architectures, and various tasks (e.g., image classification and object localization) demonstrate the superiority of our method over other methods. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

This paper proposes a novel structural dropout method CorrDrop to regularize CNNs by dropping feature units based on feature correlation, which can focus on discriminative information and improve the performance of CNNs.