Journal
ANNALS OF STATISTICS
Volume 46, Issue 3, Pages 1352-1382Publisher
INST MATHEMATICAL STATISTICS-IMS
DOI: 10.1214/17-AOS1587
Keywords
Divide and conquer; debiasing; massive data; thresholding
Categories
Funding
- NIH [2R01-GM072611-11]
- NSF [DMS-1206464, DMS-1406266]
- Direct For Mathematical & Physical Scien
- Division Of Mathematical Sciences [1406266] Funding Source: National Science Foundation
- Division Of Mathematical Sciences
- Direct For Mathematical & Physical Scien [1206464] Funding Source: National Science Foundation
Ask authors/readers for more resources
This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood-based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from k subsamples of size n/k, where n is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large k can be, as n grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available