Penalized Least Square in Sparse Setting with Convex Penalty and Non Gaussian Errors

This paper consider the penalized least squares estimators with convex penalties or regularization norms. We provide sparsity oracle inequalities for the prediction error for a general convex penalty and for the particular cases of Lasso and Group Lasso estimators in a regression setting. The main contribution is that our oracle inequalities are established for the more general case where the observations noise is issued from probability measures that satisfy a weak spectral gap (or Poincare) inequality instead of Gaussian distributions. We illustrate our results on a heavy tailed example and a sub Gaussian one; we especially give the explicit bounds of the oracle inequalities for these two special examples.

Automated summary

This paper examines penalized least squares estimators with convex penalties or regularization norms, providing sparsity oracle inequalities for prediction errors in a general convex penalty context, as well as for specific cases like Lasso and Group Lasso estimators in regression. The main contribution is the establishment of oracle inequalities for scenarios where observation noise stems from probability measures with weak spectral gaps, as opposed to just Gaussian distributions. The results are demonstrated on heavy-tailed and sub-Gaussian examples, with explicit bounds given for these special cases.