Journal
INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE
Volume 52, Issue 4, Pages 770-784Publisher
TAYLOR & FRANCIS LTD
DOI: 10.1080/00207721.2020.1839142
Keywords
Worst case uncertainty; continuous-time reinforcement learning; unknown nonlinear systems; robust control
Categories
Funding
- CONACYT [CONACyT-A1-S-8216]
- CINVESTAV [SEP-CINVESTAV-62]
Ask authors/readers for more resources
In this paper, a novel continuous-time reinforcement learning method using neural network approximation is proposed to train algorithms with worst-case uncertainty, resulting in a more robust control policy. Experimental results demonstrate that this approach is more robust than the model-based LQR method and the well-known AC method.
Reinforcement learning (RL) is an effective method to design a robust controller for unknown nonlinear systems. Uncertainty in the worst case requires a large state-action space. Hence, it is natural to use continuous-time RL methods rather than the discretisation of the spaces. In this paper, we propose a novel continuous-time RL using neural network approximation. Our method uses worst-case uncertainty to train the continuous-time RL algorithm. The backward Euler approximation is used to approximate the time derivative of the value function. Compared with the actor-critic (AC) algorithm, our method finds the robust control policy in the presence of worst-case uncertainty by taking into account the applied actions. It is shown that the AC algorithm finds the robust controller in less episodes, but its robustness is less than the results presented by our approach. The convergence of the proposed algorithm is analysed using the contraction property and differential equation techniques. The experiments show that our approach is more robust than the model-based LQR method and the well-known AC method.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available