Path-following optimal control of autonomous underwater vehicle based on deep reinforcement learning

In this paper, optimal control is applied in the path-following control problem for the autonomous underwater vehicle (AUV) when the geometry information of path is known and will not change over time. Aiming at the problem that the optimization time is too long in one control step for complex nonlinear problems, a path-following control method based on Simplified Deep Deterministic Policy Gradient (S-DDPG) algorithm is proposed. In S-DDPG, only the reward in the current state is considered, and the future reward does not need to be predicted, which avoids generating amount of meaningless failed samples and simplifies the training process of neural networks (NNs). The training is performed and completed offline before the beginning of path-following where the NNs are directly used as the controller. The simulation results show that the S-DDPG can make the AUV complete path-following tasks and has obvious advantages compared with other methods.

Automated summary

In this paper, optimal control is used to solve the path-following control problem for autonomous underwater vehicles (AUVs) with known and static path geometry. To address the issue of long optimization time for complex nonlinear problems, a path-following control method based on Simplified Deep Deterministic Policy Gradient (S-DDPG) algorithm is proposed. S-DDPG only considers the reward in the current state, eliminating the need to predict future rewards and simplifying the training process of neural networks (NNs). Offline training is conducted before path-following, where the trained NNs are directly used as controllers. Simulation results demonstrate that S-DDPG enables AUVs to successfully perform path-following tasks and outperforms other methods.