Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles

The hybrid electric-tracked vehicles (HETVs) are usually used in both agricultural and industrial applications, while the optimal energy management is critical to fully exploit the potential of HETVs. In this article, the influence of HETVs' steering resistance on the energy distribution is specially considered to model the dynamic demand accurately. Further, a multi-state energy management strategy (EMS) based on deep deterministic policy gradient (DDPG) is proposed for a series HETV in the continuous space. A multidimensional matrix framework is proposed to extract the parameters of the actor network from a trained DDPG-based EMS. Hardware-in-the-loop (HiL) experiment is conducted to validate the real-time tractability of the proposed strategy. Results suggest that the DDPG-based strategy improves the fuel economy remarkably by 13.1% and shows a more robust performance, compared with the double deep $Q$ -learning-based strategy. Though the proposed strategy is trained based on the fixed state of charge (SOC), it still exhibits a strong adaptability to the uncertainty of initial SOCs.

Automated summary

This article focuses on the energy management of hybrid electric-tracked vehicles (HETVs) used in agriculture and industry. The influence of steering resistance on energy distribution is considered, and a multi-state energy management strategy (EMS) based on deep deterministic policy gradient (DDPG) is proposed. A multidimensional matrix framework is used to extract parameters of the actor network from a trained DDPG-based EMS. Hardware-in-the-loop (HiL) experiments validate the real-time tractability of the proposed strategy. Results show that the DDPG-based strategy improves fuel economy by 13.1% compared to the double deep Q-learning-based strategy and exhibits adaptability to uncertainty in initial state of charge (SOC).