4.6 Article

Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators With Massive Data

期刊

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
卷 117, 期 537, 页码 265-276

出版社

TAYLOR & FRANCIS INC
DOI: 10.1080/01621459.2020.1773832

关键词

Big data; Distributed subsampling; Poisson sampling; Quasi-likelihood

资金

  1. Beijing Institute of Technology Research Fund Program for Young Scholars
  2. NSF [1812013]
  3. NSFC [11671019]
  4. LMEQF
  5. Division Of Mathematical Sciences
  6. Direct For Mathematical & Physical Scien [1812013] Funding Source: National Science Foundation

向作者/读者索取更多资源

This article proposes a nonuniform subsampling method based on Poisson subsampling to address the computational burden issue for massive data. The optimal subsampling probabilities are derived under the quasi-likelihood estimation and their consistency and asymptotic normality are established. A distributed subsampling framework is also developed for handling data stored in different locations. The effectiveness of the proposed methods is demonstrated through numerical experiments on simulated and real datasets.
Nonuniform subsampling methods are effective to reduce computational burden and maintain estimation efficiency for massive data. Existing methods mostly focus on subsampling with replacement due to its high computational efficiency. If the data volume is so large that nonuniform subsampling probabilities cannot be calculated all at once, then subsampling with replacement is infeasible to implement. This article solves this problem using Poisson subsampling. We first derive optimal Poisson subsampling probabilities in the context of quasi-likelihood estimation under the A- and L-optimality criteria. For a practically implementable algorithm with approximated optimal subsampling probabilities, we establish the consistency and asymptotic normality of the resultant estimators. To deal with the situation that the full data are stored in different blocks or at multiple locations, we develop a distributed subsampling framework, in which statistics are computed simultaneously on smaller partitions of the full data. Asymptotic properties of the resultant aggregated estimator are investigated. We illustrate and evaluate the proposed strategies through numerical experiments on simulated and real datasets. Supplementary materials for this article are available online.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.6
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据