Journal
SIAM JOURNAL ON MATHEMATICS OF DATA SCIENCE
Volume 2, Issue 2, Pages 480-504Publisher
SIAM PUBLICATIONS
DOI: 10.1137/19M1287365
Keywords
linear models; graphs; high-dimensional statistics
Categories
Funding
- NSF [0353079, 1447449, 1740707, 1839338, DMS 1407028]
- NIH [1U54 AI117924-01, R01 GM131381-01]
- Direct For Computer & Info Scie & Enginr
- Div Of Information & Intelligent Systems [1447449] Funding Source: National Science Foundation
- Division of Computing and Communication Foundations
- Direct For Computer & Info Scie & Enginr [0353079] Funding Source: National Science Foundation
- Division Of Mathematical Sciences
- Direct For Mathematical & Physical Scien [1839338] Funding Source: National Science Foundation
Ask authors/readers for more resources
Sparse models for high-dimensional linear regression and machine learning have received substantial attention over the past two decades. Model selection, or determining which features or covariates are the best explanatory variables, is critical to the interpretability of a learned model. Much of the current literature assumes that covariates are only mildly correlated. However, in many modern applications covariates are highly correlated and do not exhibit key properties (such as the restricted eigenvalue condition, restricted isometry property, or other related assumptions). This work considers a high-dimensional regression setting in which a graph governs both correlations among the covariates and the similarity among regression coefficients-meaning there is alignment between the covariates and regression coefficients. Using side information about the strength of correlations among features, we form a graph with edge weights corresponding to pairwise covariances. This graph is used to define a graph total variation regularizer that promotes similar weights for correlated features. This work shows how the proposed graph-based regularization yields mean-squared error guarantees for a broad range of covariance graph structures. These guarantees are optimal for many specific covariance graphs, including block and lattice graphs. Our proposed approach outperforms other methods for highly correlated design in a variety of experiments on synthetic data and real biochemistry data.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available