Journal
ANNALS OF STATISTICS
Volume 41, Issue 3, Pages 1111-1141Publisher
INST MATHEMATICAL STATISTICS
DOI: 10.1214/13-AOS1096
Keywords
Regularized regression; lasso; interactions; hierarchical sparsity; convexity
Categories
Funding
- Gerald J. Lieberman Fellowship
- NSF [DMS-09-06801, DMS-99-71405]
- AFOSR [113039]
- National Institutes of Health [N01-HV-28183]
- Direct For Mathematical & Physical Scien
- Division Of Mathematical Sciences [1208164] Funding Source: National Science Foundation
Ask authors/readers for more resources
We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting saved by the hierarchy constraint. We distinguish between parameter sparsity-the number of nonzero coefficients-and practical sparsity-the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available