期刊
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
卷 32, 期 3, 页码 1037-1051出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2020.2980004
关键词
Uncertainty; Bayes methods; Machine learning; Training; Biological neural networks; Measurement uncertainty; Bayesian statistics; convolutional neural networks (CNNs); deep learning; model uncertainty; parameter correlations; variational inference
类别
资金
- ECSEL Joint Undertaking (JU) [783163]
- European Union's Horizon 2020 Research and Innovation Program
This article presents a novel approach for training deep neural networks using Bayesian techniques, which allows for easy evaluation of model uncertainty and robustness to overfitting. The proposed method outperforms other Bayesian methods in terms of predictive accuracy and uncertainty estimation.
In this article, a novel approach for training deep neural networks using Bayesian techniques is presented. The Bayesian methodology allows for an easy evaluation of model uncertainty and, additionally, is robust to overfitting. These are commonly the two main problems classical, i.e., non-Bayesian architectures have to struggle with. The proposed approach applies variational inference in order to approximate the intractable posterior distribution. In particular, the variational distribution is defined as the product of multiple multivariate normal distributions with tridiagonal covariance matrices. Every single normal distribution belongs either to the weights or to the biases corresponding to one network layer. The layerwise a posteriori variances are defined based on the corresponding expectation values, and furthermore, the correlations are assumed to be identical. Therefore, only a few additional parameters need to be optimized compared with non-Bayesian settings. The performance of the new approach is evaluated and compared with other recently developed Bayesian methods. Basis of the performance evaluations are the popular benchmark data sets MNIST and CIFAR-10. Among the considered approaches, the proposed one shows the best predictive accuracy. Moreover, extensive evaluations of the provided prediction uncertainty information indicate that the new approach often yields more useful uncertainty estimates than the comparison methods.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据