期刊
JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION
卷 41, 期 -, 页码 406-413出版社
ACADEMIC PRESS INC ELSEVIER SCIENCE
DOI: 10.1016/j.jvcir.2016.11.003
关键词
Convolutional neural network (CNN); Nonlinear activation; RECOS model; Rectified linear unit (ReLU); MNIST dataset
资金
- DARPA
- Air Force Research Laboratory (AFRL) [FA8750-16-2-0173]
This work attempts to address two fundamental questions about the structure of the convolutional neural networks (CNN): (1) why a nonlinear activation function is essential at the filter output of all intermediate layers? (2) what is the advantage of the two-layer cascade system over the one-layer system? A mathematical model called the REctified-COrrelations on a Sphere (RECOS) is proposed to answer these two questions. After the CNN training process, the converged filter weights define a set of anchor vectors in the RECOS model. Anchor vectors represent the frequently occurring patterns (or the spectral components). The necessity of rectification is explained using the RECOS model. Then, the behavior of a two-layer RECOS system is analyzed and compared with its one-layer counterpart. The LeNet-5 and the MNIST dataset are used to illustrate discussion points. Finally, the RECOS model is generalized to a multilayer system with the AlexNet as an example. (C) 2016 Elsevier Inc. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据