Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle

An efficient energy split among different source of energy has been a challenge for existing hybrid electric vehicle (HEV) supervisory control system. It requires an optimized energy use of internal combustion engine and electric source such as battery, fuel cell, ultracapacitor, etc. In recent years, Reinforcement Learning (RL) based energy management strategy (EMS) has emerged as one of the efficient control strategies. The effectiveness Reinforcement Learning method largely depends on optimized parameter selections. However, a thorough parametric study still lacks in this field. It is a fundamental step for efficient implementation of the RL-based EMS. Different from existing RL-based EMS literature, this study conducts a parametric study on several key factors during the RL-based EMS development, including: (1) state types and number of states, (2) states and action discretization, (3) exploration and exploitation, and (4) learning experience selection. The main results show that learning experience selection can effectively reduce the vehicle fuel consumption. The study of the states and action discretization show that the vehicle fuel consumption reduces as action discretization increases while increasing the states discretization is detrimental to the fuel consumption. Moreover, the increasing number of states improves fuel economy. With the help of the proposed parametric analysis, the RL-based EMS can be easily adapted to other power split problems in a HEV application.