Estimating and Accounting for Unobserved Covariates in High-Dimensional Correlated Data

Many high-dimensional and high-throughput biological datasets have complex sample correlation structures, which include longitudinal and multiple tissue data, as well as data with multiple treatment conditions or related individuals. These data, as well as nearly all high-throughput omic data, are influenced by technical and biological factors unknown to the researcher, which, if unaccounted for, can severely obfuscate estimation of and inference on the effects of interest. We therefore developed CBCV and CorrConf: provably accurate and computationally efficient methods to choose the number of and estimate latent confounding factors present in high-dimensional data with correlated or nonexchangeable residuals. We demonstrate each method's superior performance compared to other state of the art methods by analyzing simulated multi-tissue gene expression data and identifying sex-associated DNA methylation sites in a real, longitudinal twin study.for this article are available online.

Automated summary

This article presents two methods, CBCV and CorrConf, for handling high-dimensional biological datasets with complex sample correlation structures. These methods demonstrate superior performance in choosing the number of latent confounding factors and estimating them, as evidenced by analysis of simulated and real data applications.