期刊
PATTERN RECOGNITION
卷 86, 期 -, 页码 224-235出版社
ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2018.08.017
关键词
Deep networks; Overfitting; Decorrelation
资金
- Australian Research Council [DE-180101438]
This paper proposes a novel regularizer named Structured Decorrelation Constraint, to address both the generalization and optimization of deep neural networks, including multiple-layer perceptrons and convolutional neural networks. Our proposed regularizer reduces overfitting by breaking the co-adaptions between the neurons with an explicit penalty. As a result, the network is capable of learning non-redundant representations. Meanwhile, the proposed regularizer encourages the networks to learn structured high-level features to aid the networks' optimization during training. To this end, neurons are constrained to behave obeying a group prior. Our regularizer applies to various types of layers, including fully connected layers, convolutional layers and normalization layers. The loss of our regularizer can be directly minimized along with the network's classification loss by stochastic gradient descent. Experiments show that the proposed regularizer obviously relieves the overfitting problem of the existing deep networks. It yields much better performance on extensive datasets than the conventional regularizers like Dropout. (C) 2018 Elsevier Ltd. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据