Off-Policy Learning-Based Following Control of Cooperative Autonomous Vehicles Under Distributed Attacks

This paper investigates the resilient distributed secure output path following control problem of heterogeneous autonomous ground vehicles (AGVs) subject to cyber attacks based on reinforcement learning algorithm. Most existing results are subject to the same attack models for all communication channels, however multiple channels launched by different attackers are considered in this paper. First, a predictor-acknowledgement clock algorithm for each vehicle is proposed to judge whether the communication channel among neighboring vehicles is attacked or not by receiving or transmitting an acknowledgement. Then, a resilient distributed predictor is proposed to predict the pinning vehicle's state for each vehicle. In addition, a resilient local control protocol consisting of the feedforward state provided by the predictor and the local feedback state of each vehicle is developed for the output path following problem, which is further converted to the optimal control problem by designing a discounted performance function. Discounted algebraic Riccati equations (AREs) are derived to address the optimal control problem. An off-policy reinforcement learning (RL) algorithm is put forward to learn the solution of discounted AREs online without any prior knowledge of vehicles' dynamics. It is shown that the RL-based output path following control problem of AGVs imposed by cyber attacks can be achieved in an optimal manner. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Automated summary

This paper investigates the resilient distributed secure output path following control problem of heterogeneous autonomous ground vehicles (AGVs) subject to cyber attacks based on reinforcement learning algorithm. The study considers multiple communication channels launched by different attackers and proposes a predictor-acknowledgement clock algorithm to judge the attacked communication channels. A resilient distributed predictor and a resilient local control protocol are developed for predicting and controlling the path following problem. The optimal control problem is solved using discounted algebraic Riccati equations (AREs) and an off-policy reinforcement learning (RL) algorithm is proposed to learn the solution online.