Graph attention mechanism based reinforcement learning for multi-agent flocking control in communication-restricted environment

To solve the poor performance of reinforcement learning (RL) in the multi-agent flocking cooperative control under the communication-restricted environments, we propose a multi-agent cooperative RL (MACRL) method based on the equivalent characteristics of the agents in the flocking task. A distance graph attention (GAT) mechanism is introduced into the policy network of the proposed MACRL to change the agent's attention weights related to neighbors and reduce the influence of remote neighbors which have poor com-munication quality on the agent's behavioral decision-making. Furthermore, a distance GAT-based MACRL (DGAT-MACRL) algorithm is proposed for multi-agent flocking control in the communication-restricted environment. The simulation results show that the pro-posed flocking algorithm has good adaptability to the communication delay and the com-munication distance constraint environments. The flocking control effect is significantly better than other RL-based flocking algorithms and traditional flocking algorithms. In addition, the good experimental performance has also confirmed that the proposed DGAT-MACRL is an effective solution to the problem of how to improve the adaptability of traditional RL to the flocking control system with a dynamic scale. Our algorithm provides a novel and practical algorithm offering an effective method for the cooperative task accomplished by multi-agent in non-ideal environments.(c) 2022 Elsevier Inc. All rights reserved.

Automated summary

To address the poor performance of reinforcement learning (RL) in multi-agent flocking cooperative control under communication-restricted environments, a distance graph attention (GAT) mechanism is introduced into a multi-agent cooperative RL (MACRL) method. This mechanism changes the attention weights of agents in the flocking task related to neighbors and reduces the influence of remote neighbors with poor communication quality on the agent's behavioral decision-making. Furthermore, a distance GAT-based MACRL (DGAT-MACRL) algorithm is proposed for multi-agent flocking control in communication-restricted environments.