Receding-Horizon Reinforcement Learning Approach for Kinodynamic Motion Planning of Autonomous Vehicles

Kinodynamic motion planning is critical for autonomous vehicles with high maneuverability in dynamic environments. However, obtaining near-optimal motion planning solutions with low computational costs and inaccurate prior model information is challenging. To address this issue, this paper proposes a receding-horizon reinforcement learning approach for kinodynamic motion planning (RHRL-K DP) of autonomous vehicles in the presence of inaccurate dynamics information and moving obstacles. Specifically, a receding-horizon actor-critic reinforcement learning algorithm is presented, resulting in a neural network-based planning strategy that can be learned both offline and online. A neural network-based model is built and learned online to approximate the modeling uncertainty of the prior nominal model in order to improve planning performance. Furthermore, active collision avoidance in dynamic environments is realized by constructing safety-related terms in actor and critic networks using potential fields. In theory, the uniformly ultimate boundedness property of the modeling uncertainty's approximation error is proven, and the convergence of the proposed RHRL-KDP is analyzed. Simulation tests show that our approach outperforms the previously developed motion planners based on model predictive control (MPC), safe RL, and RRT* in terms of planning performance. Furthermore, in both online and offline learning scenarios, RHRL-KDP outperforms MPC and RRT* in terms of computational efficiency.

Automated summary

This paper proposes a kinodynamic motion planning approach for autonomous vehicles using a receding-horizon reinforcement learning algorithm. The algorithm utilizes a neural network-based planning strategy that can be learned both offline and online, and a neural network-based model is built and learned online to improve planning performance. Simulation tests demonstrate the superior planning performance and computational efficiency of the proposed approach compared to previous motion planners.