Journal
IEEE ACCESS
Volume 7, Issue -, Pages 9540-9557Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/ACCESS.2018.2890740
Keywords
Feedforward neural networks; pruning hidden layer nodes and weights; group L-1(/2); smooth group L-1/2; group lasso; convergence
Categories
Funding
- Natural Science Foundation of China [61473059, 11401076, 61473328]
- Fundamental Research Funds for the Central Universities [DUT13-RC(3) 068, DUT18JC02, DUT17LK46]
- Dalian High Level Talent Innovation Support Program [2015R057]
Ask authors/readers for more resources
A group L-1(/2) regularization term is defined and introduced into the conventional error function for pruning the hidden layer nodes of feedforward neural networks. This group L-1(/2) regularization method (GL(1/2)) can prune not only the redundant hidden nodes but also the redundant weights of the surviving hidden nodes of the neural networks. As a comparison, the popular group lasso regularization (GL(2)) can prune the redundant hidden nodes, but cannot prune any redundant weights of the surviving hidden nodes, of the neural networks. A disadvantage of the GL(1/2) is that it involves a non-smooth absolute value function, which causes oscillation in the numerical computation and difficulty in the convergence analysis. As a remedy, the absolute value function is approximated by a smooth function, resulting in a smooth group L-1(/2) regularization method (SGL(1/2)). Numerical simulations on a few benchmark data sets show that, compared with GL(2), SGL(1/2) can achieve better accuracy and remove more redundant nodes and weights of the surviving hidden nodes. A convergence theorem is also proved for SGL(1/2).
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available