Journal
MACHINE LEARNING
Volume 100, Issue 2-3, Pages 255-283Publisher
SPRINGER
DOI: 10.1007/s10994-015-5484-1
Keywords
Reinforcement learning; Markov Decision Process; Lipschitz continuity; Policy gradient algorithm
Categories
Ask authors/readers for more resources
This paper is about the exploitation of Lipschitz continuity properties for Markov Decision Processes to safely speed up policy-gradient algorithms. Starting from assumptions about the Lipschitz continuity of the state-transition model, the reward function, and the policies considered in the learning process, we show that both the expected return of a policy and its gradient are Lipschitz continuous w.r.t. policy parameters. By leveraging such properties, we define policy-parameter updates that guarantee a performance improvement at each iteration. The proposed methods are empirically evaluated and compared to other related approaches using different configurations of three popular control scenarios: the linear quadratic regulator, the mass-spring-damper system and the ship-steering control.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available