Journal
PATTERN RECOGNITION
Volume 120, Issue -, Pages -Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2021.108117
Keywords
Over-fitting; Regularization; Dropout; Convolutional neural networks
Funding
- National Key Research and Development Program of China [2018YFB1800204]
- National Natural Science Foundation of China [61771273]
- Natural Science Foundation of Guangdong Province [2021A1515011807]
- Guangdong Basic and Applied Basic Research Foundation [2019A1515110344]
- R&D Program of Shenzhen [JCYJ20180508152204044]
- China Postdoctoral Sci-ence Foundation [2021T140375]
Ask authors/readers for more resources
This paper proposes a novel structural dropout method CorrDrop to regularize CNNs by dropping feature units based on feature correlation, which can focus on discriminative information and improve the performance of CNNs.
Convolutional neural networks (CNNs) easily suffer from the over-fitting problem since they are often over-parameterized in the case of small training datasets. The conventional dropout that drops feature units randomly works well for fully connected networks, but fails to regularize CNNs well due to high spatial correlation of the intermediate features, which allows the dropped information to flow through the network, thus leading to the problem of under-dropping. To better regularize CNNs, some structural dropout methods such as SpatialDropout and DropBlock have been proposed by dropping feature units in continuous regions randomly. However, these methods may suffer from the over-dropping problem by discarding the critical discriminative features, thus limiting the performance of CNNs. To address these issues, we propose a novel structural dropout method, Correlation based Dropout (CorrDrop), to regularize CNNs by dropping feature units based on feature correlation. Unlike the previous dropout methods, our CorrDrop can focus on the discriminative information and drops features in a spatial-wise or channel wise manner. Extensive experiments on different datasets, network architectures, and various tasks (e.g., image classification and object localization) demonstrate the superiority of our method over other methods. (c) 2021 Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available