3.8 Proceedings Paper

Gradient-Free Neural Network Training Based on Deep Dictionary Learning with the Log Regularizer

期刊

PATTERN RECOGNITION AND COMPUTER VISION, PT IV
卷 13022, 期 -, 页码 561-574

出版社

SPRINGER INTERNATIONAL PUBLISHING AG
DOI: 10.1007/978-3-030-88013-2_46

关键词

Deep dictionary learning; log regularizer; Block coordinate descent; Sparse proximal operator; Gradient-free network

向作者/读者索取更多资源

This paper introduces a gradient-free neural network training method by using deep dictionary learning and logarithm function as sparse regularizer for feature extraction in network training. Proximal block coordinate descent method and log-thresholding operator are employed for optimizing non-convex and nonsmooth subproblems.
Gradient-free neural network training is attracting increasing attentions, which efficiently to avoid the gradient vanishing issue in traditional neural network training with gradient-based methods. The state-of-the-art gradient-free methods introduce a quadratic penalty or use an equivalent approximation of the activation function to achieve the training process without gradients, but they are hardly to mine effective signal features since the activation function is a limited nonlinear transformation. In this paper, we first propose to construct the neural network training as a deep dictionary learning model for achieving the gradientfree training of the network. To further enhance the ability of feature extraction in network training based on gradient-free method, we introduce the logarithm function as a sparsity regularizer which introduces accurate sparse activations on the hidden layer except for the last layer. Then, we employ a proximal block coordinate descent method to forward update the variables of each layer and apply the log-thresholding operator to achieve the optimization of the non-convex and non-smooth subproblems. Finally, numerical experiments conducted on several publicly available datasets prove the sparse representation of inputs is effective for gradient-free neural network training.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

3.8
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据