Journal
BERNOULLI
Volume 19, Issue 4, Pages 1449-1464Publisher
INT STATISTICAL INST
DOI: 10.3150/12-BEJSP15
Keywords
convex analysis; Gaussian processes; least squares; penalized regression
Categories
Funding
- NSF [DMS-12-08857]
- AFOSR [113039]
- Direct For Mathematical & Physical Scien
- Division Of Mathematical Sciences [1208857] Funding Source: National Science Foundation
Ask authors/readers for more resources
It has been over 200 years since Gauss's and Legendre's famous priority dispute on who discovered the method of least squares. Nevertheless, we argue that the normal equations are still relevant in many facets of modern statistics, particularly in the domain of high-dimensional inference. Even today, we are still learning new things about the law of large numbers, first described in Bernoulli's Ars Conjectandi 300 years ago, as it applies to high dimensional inference. The other insight the normal equations provide is the asymptotic Gaussianity of the least squares estimators. The general form of the Gaussian distribution, Gaussian processes, are another tool used in modern high-dimensional inference. The Gaussian distribution also arises via the central'limit theorem in describing weak convergence of the usual least squares estimators. In terms of high-dimensional inference, we are still missing the right notion of weak convergence. In this mostly expository work, we try to describe how both the normal equations and the theory of Gaussian processes, what we refer to as the geometry of least squares, apply to many questions of current interest.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available