Safe Reinforcement Learning Control for Water Distribution Network

Reinforcement Learning (RL) is an optimal control method for regulating the behaviour of a dynamical system when the system model is unknown. This feature is a strong advantage for controlling systems, such as Water Distribution Networks, where it is difficult to have a reliable model. When learning an optimal policy with RL, the exploration phase implies high degree of uncertainty in the system operation. Large scale infrastructures such as WDN require a robust operation since they cannot afford fails during the operation. This paper presents a model-free control method which provides safety in the operation while learning an optimal policy. This method introduces a policy supervisor block in the control loop which assesses the safety of the learned policy in real-time. The safety verification consists of evaluating the trajectory on a standard linear model. In this model only the fundamental linear dynamics are represented and the system's dimensions do not require to be expressed with high accuracy. If the predicted trajectory violates the boundaries, the supervisor provides a safe control action. Simulation and experimental results prove the applicability of the proposed method.

Automated summary

Reinforcement Learning (RL) is an optimal control method used for regulating the behavior of dynamical systems with unknown models. The proposed model-free control method introduces a policy supervisor block in the control loop to ensure safety in operation while learning the optimal policy, by evaluating trajectories on a standard linear model in real-time. Simulation and experimental results show the effectiveness of this method in ensuring robust operation in large-scale infrastructures like Water Distribution Networks.