Journal
NEUROCOMPUTING
Volume 458, Issue -, Pages 490-499Publisher
ELSEVIER
DOI: 10.1016/j.neucom.2021.06.067
Keywords
Convolutional neural network; Nonmonotonic activation function; Logish; Image classification; Top-1 accuracy
Categories
Funding
- Natural Science Foundation of Liaoning Province [2020-MS-080]
- Fundamental Research Funds for the Central Universities [N2005032]
- National Natural Science Foundation of China [61772125]
Ask authors/readers for more resources
In this paper, a new nonlinear nonmonotonic activation function called Logish is proposed, demonstrating better performance in image classification tasks compared to other common activation functions.
Activation function is an important component of the convolutional neural network. Recently, nonlinear nonmonotonic activation functions such as Swish and Mish have illustrated good performance in deep learning structures. In this paper, we propose a new nonlinear nonmonotonic activation function called Logish, which can be represented by f (x) = x . ln[1 + sigmoid (x)]. Firstly, we take the logarithmic operation to reduce the numerical range of sigmoid (x) + 1, then we employ variable x to make the negative output have a strong regularization effect. Furthermore, we evaluate the image classification performance of Logish and its variant f (x) = alpha x . ln[1 + sigmoid (beta x)] in simple and complex networks with top-1 accuracy. Experimental results demonstrate that Logish's variant (alpha = 1; beta = 10) can achieve 94.8% top-1 accuracy with ResNet-50 network on CIFAR 10 dataset, and can reach 99.24% top-1 accuracy with DenseNet on MNIST dataset and 88.52% top-1 accuracy with SE-Inception-v4 network on SVHN dataset respectively. It is higher than the Sigmoid, Tanh, ReLU, Swish and Mish activation functions in the corresponding dataset. It also verifies the performance and effectiveness of Logish. (C) 2021 Elsevier B.V. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available