Gradient-Free Neural Network Training Based on Deep Dictionary Learning with the Log Regularizer

Article Computer Science, Artificial Intelligence

Training Neural Networks by Lifted Proximal Operator Machines

Jia Li et al.

Summary: LPOM is a method for training fully-connected feed-forward neural networks that represents the activation function as an equivalent proximal operator and applies block multi-convex handling to all layer-wise weights and activations. It avoids gradient vanishing or exploding issues and is memory-efficient with relatively easy parameter tuning.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

添加到收藏夹

Article Automation & Control Systems

Accelerated Log-Regularized Convolutional Transform Learning and its Convergence Guarantee

Zhenni Li et al.

Summary: The article introduces a new CTL framework with a log regularizer for accurate representations and strong sparsity. By employing PDCA algorithm and extrapolation technology, the algorithm is accelerated for fast and efficient CTL learning. The experimental results show stable convergence with lower approximation error and faster speed compared to existing CTL algorithms.

IEEE TRANSACTIONS ON CYBERNETICS (2022)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Cross-Iteration Batch Normalization

Zhuliang Yao et al.

Summary: The proposed Cross-Iteration Batch Normalization (CBN) addresses the reduced effectiveness of batch normalization in small mini-batch sizes by jointly utilizing examples from multiple recent iterations to enhance estimation quality. A compensation technique based on Taylor polynomials is introduced to account for network weight changes, allowing for accurate estimation of statistics and effective application of batch normalization. CBN outperforms the original batch normalization and a direct calculation of statistics over previous iterations in object detection and image classification tasks with small mini-batch sizes.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

添加到收藏夹

Proceedings Paper Acoustics

INERTIAL PROXIMAL DEEP LEARNING ALTERNATING MINIMIZATION FOR EFFICIENT NEUTRAL NETWORK TRAINING

Linbo Qiao et al.

Summary: In recent years, a new algorithm called iPDLAM has been developed to improve the Deep Learning Alternating Minimization (DLAM) by using the inertial technique to predict points more accurately and applying a warm-up technique to accelerate training speed. Numerical results on real-world datasets have demonstrated the efficiency of the proposed algorithm.

2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021) (2021)

添加到收藏夹

Proceedings Paper Acoustics

DYNAMIC SPARSITY NEURAL NETWORKS FOR AUTOMATIC SPEECH RECOGNITION

Zhaofeng Wu et al.

Summary: This paper introduces Dynamic Sparsity Neural Networks (DSNN), which can instantly switch to any predefined sparsity configuration at run-time, demonstrating its effectiveness and flexibility through experiments on Google Voice Search data, showing that its performance is comparable to individually trained single sparsity networks.

2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021) (2021)

添加到收藏夹

Article Computer Science, Artificial Intelligence

More trainable inception-ResNet for face recognition

Shuai Peng et al.

NEUROCOMPUTING (2020)

添加到收藏夹

Proceedings Paper Computer Science, Information Systems

ADMM for Efficient Deep Learning with Global Convergence

Junxiang Wang et al.

KDD'19: PROCEEDINGS OF THE 25TH ACM SIGKDD INTERNATIONAL CONFERENCCE ON KNOWLEDGE DISCOVERY AND DATA MINING (2019)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Dict Layer: A Structured Dictionary Layer

Yefei Chen et al.

PROCEEDINGS 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW) (2018)

添加到收藏夹

Article Geochemistry & Geophysics

Discriminative Robust Deep Dictionary Learning for Hyperspectral Image Classification

Vanika Singhal et al.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2017)

添加到收藏夹

Gradient-Free Neural Network Training Based on Deep Dictionary Learning with the Log Regularizer

相关参考文献

导出引文

分享论文