An AUV Target-Tracking Method Combining Imitation Learning and Deep Reinforcement Learning

This study aims to solve the problem of sparse reward and local convergence when using a reinforcement learning algorithm as the controller of an AUV. Based on the generative adversarial imitation (GAIL) algorithm combined with a multi-agent, a multi-agent GAIL (MAG) algorithm is proposed. The GAIL enables the AUV to directly learn from expert demonstrations, overcoming the difficulty of slow initial training of the network. Parallel training of multi-agents reduces the high correlation between samples to avoid local convergence. In addition, a reward function is designed to help training. Finally, the results show that in the unity simulation platform test, the proposed algorithm has a strong optimal decision-making ability in the tracking process.

Automated summary

This study aims to address the issues of sparse reward and local convergence in using a reinforcement learning algorithm as an AUV controller. By combining the GAIL algorithm with multi-agent approach, the MAG algorithm is proposed, allowing the AUV to learn directly from expert demonstrations and overcoming the slow initial training problem. The parallel training of multi-agents reduces sample correlation and prevents local convergence. Additionally, a reward function is designed to assist in training. The results of the Unity simulation platform test demonstrate the strong decision-making ability of the proposed algorithm in the tracking process.