期刊
NEUROCOMPUTING
卷 458, 期 -, 页码 490-499出版社
ELSEVIER
DOI: 10.1016/j.neucom.2021.06.067
关键词
Convolutional neural network; Nonmonotonic activation function; Logish; Image classification; Top-1 accuracy
资金
- Natural Science Foundation of Liaoning Province [2020-MS-080]
- Fundamental Research Funds for the Central Universities [N2005032]
- National Natural Science Foundation of China [61772125]
In this paper, a new nonlinear nonmonotonic activation function called Logish is proposed, demonstrating better performance in image classification tasks compared to other common activation functions.
Activation function is an important component of the convolutional neural network. Recently, nonlinear nonmonotonic activation functions such as Swish and Mish have illustrated good performance in deep learning structures. In this paper, we propose a new nonlinear nonmonotonic activation function called Logish, which can be represented by f (x) = x . ln[1 + sigmoid (x)]. Firstly, we take the logarithmic operation to reduce the numerical range of sigmoid (x) + 1, then we employ variable x to make the negative output have a strong regularization effect. Furthermore, we evaluate the image classification performance of Logish and its variant f (x) = alpha x . ln[1 + sigmoid (beta x)] in simple and complex networks with top-1 accuracy. Experimental results demonstrate that Logish's variant (alpha = 1; beta = 10) can achieve 94.8% top-1 accuracy with ResNet-50 network on CIFAR 10 dataset, and can reach 99.24% top-1 accuracy with DenseNet on MNIST dataset and 88.52% top-1 accuracy with SE-Inception-v4 network on SVHN dataset respectively. It is higher than the Sigmoid, Tanh, ReLU, Swish and Mish activation functions in the corresponding dataset. It also verifies the performance and effectiveness of Logish. (C) 2021 Elsevier B.V. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据