Random weighting in LASSO regression

We establish statistical properties of random-weighting methods in LASSO regression under different regularization parameters lambda(n) and suitable regularity conditions. The random-weighting methods in view concern repeated optimization of a randomized objective function, motivated by the need for computationally efficient uncertainty quantification in contemporary estimation settings. In the context of LASSO regression, we repeatedly assign analyst-drawn random weights to terms in the objective function, and optimize to obtain a sample of random-weighting estimators. We show that existing approaches have conditional model selection consistency and conditional asymptotic normality at different growth rates of lambda(n) as n -> infinity. We propose an extension to the available random-weighting methods and establish that the resulting samples attain conditional sparse normality and conditional consistency in a growing-dimension setting. We illustrate the proposed methodology using synthetic and benchmark data sets, and we discuss the relationship of the results to approximate nonparametric Bayesian analysis and to perturbation bootstrap methods.

Automated summary

This article establishes the statistical properties of random-weighting methods in LASSO regression under different regularization parameters and suitable regularity conditions. By assigning random weights to terms in the objective function and optimizing them, we obtain a set of random-weighting estimators. The study shows that existing methods have conditional model selection consistency and conditional asymptotic normality at different growth rates. Moreover, an extension to these methods is proposed, demonstrating conditional sparse normality and consistency in a growing-dimension setting. The effectiveness of the proposed methodology is illustrated using synthetic and benchmark datasets, and its relationship to approximate nonparametric Bayesian analysis and perturbation bootstrap methods is discussed.