期刊
AUTOMATICA
卷 50, 期 2, 页码 475-489出版社
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.automatica.2013.12.009
关键词
LQR; Generalized policy iteration; Reinforcement learning; Adaptive control; Optimization under uncertainties
资金
- Institute of BioMed-IT, Energy-IT and Smart-IT Technology (BEST), a Brain Korea 21 plus program, Yonsei University
This paper mathematically analyzes the integral generalized policy iteration (I-GPI) algorithms applied to a class of continuous-time linear quadratic regulation (LQR) problems with the unknown system matrix A. GPI is the general idea of interacting policy evaluation and policy improvement steps of policy iteration (PI), for computing the optimal policy. We first introduce the update horizon (h) over bar, and then show that (i) all of the I-GPI methods with the same (h) over bar can be considered equivalent and that (ii) the value function approximated in the policy evaluation step monotonically converges to the exact one as (h) over bar -> infinity. This reveals the relation between the computational complexity and the update (or time) horizon of I-GPI as well as between I-PI and I-GPI in the limit (h) over bar -> infinity. We also provide and discuss two modes of convergence of I-GPI; I-GPI behaves like PI in one mode, and in the other mode, it performs like value iteration for discrete-time LQR and infinitesimal GPI ((h) over bar -> 0). From these results, a new classification of the integral reinforcement learning is formed with respect to (h) over bar. Two matrix inequality conditions for stability, the region of local monotone convergence, and data-driven (adaptive) implementation methods are also provided with detailed discussion. Numerical simulations are carried out for verification and further investigations. (C) 2013 Elsevier Ltd. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据