4.7 Article

Stochastic quasi-Newton with line-search regularisation

Journal

AUTOMATICA
Volume 127, Issue -, Pages -

Publisher

PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.automatica.2021.109503

Keywords

Nonlinear system identification; Stochastic optimisation; Stochastic gradient; Stochastic quasi-Newton; Sequential Monte Carlo; Particle filter; Gaussian process

Funding

  1. Swedish Research Council, Sweden [621-2016-06079, 2017-03807]
  2. Swedish Foundation for Strategic Research (SSF), Sweden [RIT15-0012]
  3. Swedish Research Council [2017-03807] Funding Source: Swedish Research Council
  4. Swedish Foundation for Strategic Research (SSF) [RIT15-0012] Funding Source: Swedish Foundation for Strategic Research (SSF)

Ask authors/readers for more resources

This paper introduces a novel quasi-Newton algorithm for stochastic optimization, extending the rapid convergence and computational attractiveness seen in deterministic optimization problems. The algorithm learns second-order information based on observing first-order gradients, with a highly flexible model for the Hessian. A stochastic counterpart to standard line-search procedures is proposed and demonstrated to be useful in maximum likelihood identification for general nonlinear state space models.
In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (Hessian) information based on observing first-order gradients. We extend these ideas to the stochastic setting by employing a highly flexible model for the Hessian and infer its value based on observing noisy gradients. In addition, we propose a stochastic counterpart to standard line-search procedures and demonstrate the utility of this combination on maximum likelihood identification for general nonlinear state space models. (C) 2021 Elsevier Ltd. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available