Journal
ANNALS OF STATISTICS
Volume 49, Issue 3, Pages 1378-1406Publisher
INST MATHEMATICAL STATISTICS-IMS
DOI: 10.1214/20-AOS2004
Keywords
Robust optimization; minimax optimality; risk-averse learning
Categories
Funding
- SAIL-Toyota Center for AI Research
- National Science Foundation [NSF-CAREER-1553086]
- Office of Naval Research YIP Award [N00014-19-2288]
- Samsung Fellowship
Ask authors/readers for more resources
A distributionally robust stochastic optimization framework is developed and analyzed, which learns a model providing good performance against perturbations to the data-generating distribution, with a convex formulation for the problem and convergence guarantees. Finite-sample minimax upper and lower bounds are proved, showing that distributional robustness sometimes comes at a cost in convergence rates. Limit theorems for the learned parameters are provided, with fully specified limiting distribution for computing confidence intervals.
A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts or unmodeled temporal effects. We develop and analyze a distributionally robust stochastic optimization (DRO) framework that learns a model providing good performance against perturbations to the data-generating distribution. We give a convex formulation for the problem, providing several convergence guarantees. We prove finite-sample minimax upper and lower bounds, showing that distributional robustness sometimes comes at a cost in convergence rates. We give limit theorems for the learned parameters, where we fully specify the limiting distribution so that confidence intervals can be computed. On real tasks including generalizing to unknown subpopulations, fine-grained recognition and providing good tail performance, the distributionally robust approach often exhibits improved performance.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available