Control of Hybrid Electric Vehicle Powertrain Using Offline-Online Hybrid Reinforcement Learning

Hybrid electric vehicles can achieve better fuel economy than conventional vehicles by utilizing multiple power sources. While these power sources have been controlled by rule-based or optimization-based control algorithms, recent studies have shown that machine learning-based control algorithms such as online Deep Reinforcement Learning (DRL) can effectively control the power sources as well. However, the optimization and training processes for the online DRL-based powertrain control strategy can be very time and resource intensive. In this paper, a new offline-online hybrid DRL strategy is presented where offline vehicle data are exploited to build an initial model and an online learning algorithm explores a new control policy to further improve the fuel economy. In this manner, it is expected that the agent can learn an environment consisting of the vehicle dynamics in a given driving condition more quickly compared to the online algorithms, which learn the optimal control policy by interacting with the vehicle model from zero initial knowledge. By incorporating a priori offline knowledge, the simulation results show that the proposed approach not only accelerates the learning process and makes the learning process more stable, but also leads to a better fuel economy compared to online only learning algorithms.

Automated summary

Hybrid electric vehicles can achieve better fuel economy by utilizing multiple power sources. Recent studies have shown that machine learning-based control algorithms, such as online Deep Reinforcement Learning (DRL), can effectively control these power sources. However, the optimization and training processes for the online DRL-based control strategy can be time and resource intensive. This paper presents a new offline-online hybrid DRL strategy that uses offline vehicle data to build an initial model and an online learning algorithm to improve fuel economy.