4.6 Article

Fluctuations, bias, variance and ensemble of learners: exact asymptotics for convex losses in high-dimension

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Review Computer Science, Artificial Intelligence

A review on weight initialization strategies for neural networks

Meenal V. Narkhede et al.

Summary: Neural network weight initialization is a critical step that directly affects the convergence of the network. This paper discusses various weight initialization techniques, classifies them, and provides an overview of different techniques in the field.

ARTIFICIAL INTELLIGENCE REVIEW (2022)

Article Mathematics, Applied

The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve

Song Mei et al.

Summary: Deep learning methods defy traditional statistical mindset with rich neural network architectures that achieve small generalization errors on real data. Research shows that the global minimum test error is often found in extreme overparametrization regime, above the interpolation threshold.

COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS (2022)

Article Mechanics

Triple descent and the two kinds of overfitting: where and why do they appear?*

Stephane d'Ascoli et al.

Summary: This paper discusses the "double descent" phenomenon in deep learning and the difference between nonlinear and linear peaks, providing theoretical analysis and comparisons of results from numerical experiments involving deep neural networks.

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT (2021)

Article Mechanics

Scaling description of generalization with number of parameters in deep learning

Mario Geiger et al.

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT (2020)

Article Multidisciplinary Sciences

Benign overfitting in linear regression

Peter L. Bartlett et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2020)

Letter Multidisciplinary Sciences

REPLY TO LOOG ET AL.: Looking beyond the peaking phenomenon

Mikhail Belkin et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2020)

Article Physics, Multidisciplinary

Modeling the Influence of Data Structure on Learning in Neural Networks: The Hidden Manifold Model

Sebastian Goldt et al.

PHYSICAL REVIEW X (2020)

Article Mathematics, Applied

State evolution for approximate message passing with non-separable functions

Raphael Berthier et al.

INFORMATION AND INFERENCE-A JOURNAL OF THE IMA (2020)

Article Physics, Multidisciplinary

A jamming transition from under- to over-parametrization affects generalization in deep learning

S. Spigler et al.

JOURNAL OF PHYSICS A-MATHEMATICAL AND THEORETICAL (2019)

Article Computer Science, Information Systems

Precise Error Analysis of Regularized M-Estimators in High Dimensions

Christos Thrampoulidis et al.

IEEE TRANSACTIONS ON INFORMATION THEORY (2018)

Article Statistics & Probability

High dimensional robust M-estimation: asymptotic variance via approximate message passing

David Donoho et al.

PROBABILITY THEORY AND RELATED FIELDS (2016)

Article Physics, Condensed Matter

Statistical physics of inference: thresholds and algorithms

Lenka Zdeborová et al.

ADVANCES IN PHYSICS (2016)

Article Mathematics, Applied

State evolution for general approximate message passing algorithms, with applications to spatial coupling

Adel Javanmard et al.

INFORMATION AND INFERENCE-A JOURNAL OF THE IMA (2013)

Article Computer Science, Information Systems

The LASSO Risk for Gaussian Matrices

Mohsen Bayati et al.

IEEE TRANSACTIONS ON INFORMATION THEORY (2012)

Article Computer Science, Information Systems

The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing

Mohsen Bayati et al.

IEEE TRANSACTIONS ON INFORMATION THEORY (2011)

Article Statistics & Probability

THE SPECTRUM OF KERNEL RANDOM MATRICES

Noureddine El Karoui

ANNALS OF STATISTICS (2010)

Article Multidisciplinary Sciences

Message-passing algorithms for compressed sensing

David L. Donoho et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2009)

Article Automation & Control Systems

Bregman monotone optimization algorithms

HH Bauschke et al.

SIAM JOURNAL ON CONTROL AND OPTIMIZATION (2003)