Reinforcement Learning and Deep Neural Networks for PI Controller Tuning

Reinforcement Learning, using deep neural networks, has recently gained prominence owing to its ability to train autonomous agents that have defeated human players in various complex games. Here, Reinforcement Learning is applied to the challenge of automatically tuning a proportional -integral controller, given only the process variable, set-point, manipulated variable and prior controller gains. The training considers random changes in plant dynamics, disturbances and measurement noise. Two training procedures were tested in this work, one that built up the difficulty of the simulation over time, and another that used the full complexity of the simulation throughout the training. The results show that building up the difficulty of the simulation by introducing greater degrees of randomness as the training progresses, produces an agent that is better able to tune the controller in question. Additional experience gathered in completing this work is also discussed to enable the reader to avoid some of the challenges encountered. (C) 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.