期刊
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
卷 117, 期 540, 页码 1858-1874出版社
TAYLOR & FRANCIS INC
DOI: 10.1080/01621459.2021.1891925
关键词
Distributed inference; Divide-and-conquer; Nonsmooth loss; Quantile regression; Stochastic gradient descent
资金
- National Program on Key Basic Research Project (973 Program) [2018AAA0100704]
- NSFC [11825104, 11690013]
- Youth Talent Support Program
- Australian Research Council
This article investigates distributed estimation and inference for a general statistical problem by proposing a new multi-round distributed estimation procedure to overcome restrictions on the number of machines. The method introduces a computationally efficient estimator for Sigma(-1)w, applicable to nondifferentiable losses, facilitating inference for the empirical risk minimizer.
This article studies distributed estimation and inference for a general statistical problem with a convex loss that could be nondifferentiable. For the purpose of efficient computation, we restrict ourselves to stochastic first-order optimization, which enjoys low per-iteration complexity. To motivate the proposed method, we first investigate the theoretical properties of a straightforward divide-and-conquer stochastic gradient descent approach. Our theory shows that there is a restriction on the number of machines and this restriction becomes more stringent when the dimension p is large. To overcome this limitation, this article proposes a new multi-round distributed estimation procedure that approximates the Newton step only using stochastic subgradient. The key component in our method is the proposal of a computationally efficient estimator of Sigma(-1)w, where Sigma is the population Hessian matrix and w is any given vector. Instead of estimating Sigma (or Sigma(-1)) that usually requires the second-order differentiability of the loss, the proposed first-order Newton-type estimator (FONE) directly estimates the vector of interest Sigma(-1)w as a whole and is applicable to nondifferentiable losses. Our estimator also facilitates the inference for the empirical risk minimizer. It turns out that the key term in the limiting covariance has the form of Sigma(-1)w, which can be estimated by FONE.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据