Journal
ELECTRONIC JOURNAL OF STATISTICS
Volume 1, Issue -, Pages 1-29Publisher
INST MATHEMATICAL STATISTICS
DOI: 10.1214/07-EJS004
Keywords
regression; lasso; stagewise
Categories
Funding
- National Science Foundation [DMS-0204612, DMS-0505676, DMS-9971405, DMS-0505682]
- National Institutes of Health [2R01 CA 72028-07, N01-HV-28183, 5R33HL068522]
Ask authors/readers for more resources
We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron, Hastie, Johnstone & Tibshirani (2004) it is proved that the least angle regression algorithm, with a small modification, solves the lasso regression problem. Here we give an analogous result for incremental forward stage-wise regression, showing that it solves a version of the lasso problem that enforces monotonicity. One consequence of this is as follows: while lasso makes optimal progress in terms of reducing the residual sum-of-squares per unit increase in L-1-norm of the coefficient beta, forward stage-wise is optimal per unit L-1 arc-length traveled along the coefficient path. We also study a condition under which the coefficient paths of the lasso are monotone, and hence the different algorithms coincide. Finally, we compare the lasso and forward stagewise procedures in a simulation study involving a large number of correlated predictors.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available