☆ 4.5 Article

An Efficient Asymmetric Nonlinear Activation Function for Deep Neural Networks

SYMMETRY-BASEL (2022)

Discovering Parametric Activation Functions

Garrett Bingham et al.

Summary: This paper proposes a technique for customizing activation functions automatically, resulting in reliable improvements in performance. It discovers general activation functions and specialized functions for different neural network architectures, consistently improving accuracy over ReLU and other activation functions.

NEURAL NETWORKS (2022)

添加到收藏夹

Article Engineering, Electrical & Electronic

Parametric rectified nonlinear unit (PRenu) for convolution neural networks

Ilyas El Jaafari et al.

Summary: The proposed parametric rectified nonlinear function unit (PRenu) is similar to Relu but differentiates by providing a non-linear transformation for positive values and improves CNN convergence speed and accuracy.

SIGNAL IMAGE AND VIDEO PROCESSING (2021)

添加到收藏夹

Review Computer Science, Artificial Intelligence

A survey on modern trainable activation functions

Andrea Apicella et al.

Summary: In recent years, there has been a renewed interest in trainable activation functions, which can be trained during the learning process to improve neural network performance. Various models of trainable activation functions have been proposed in the literature, many of which are equivalent to adding neuron layers with fixed activation functions and simple local rules.

NEURAL NETWORKS (2021)

添加到收藏夹

Article Computer Science, Artificial Intelligence