Q-learning based energy management strategy for a hybrid multi-stack fuel cell system considering degradation

The use of multi-stack fuel cells (FCs) is attracting considerable attention in electrified vehicles due to the added degrees of freedom in terms of efficiency and survivability. In a multi-stack FC hybrid electric vehicle, the power sources (FCs and the battery pack) have different energetic characteristics and their operation is influenced by the performance drifts caused by degradation. Hence, efficient power distribution for such a multi-source system is a critical issue. This paper proposes a three-layer online EMS for a recreational vehicle composed of three FCs and a battery pack. In the first layer, two online estimators are responsible for constantly updating the char-acteristics of each FC and the battery to be used by the power distribution algorithm. In the second layer, a rule-based method is developed to improve the calculation speed of the power distribution algorithm by deciding when it should be activated. The last layer performs the power distribution between FCs and battery using a model-free reinforcement learning (RL) algorithm called Q-learning. The proposed RL-based EMS attempts to meet the requested power while minimizing the costs of hydrogen consumption and degradation of all power sources. To justify the performance of the proposed strategy, a comprehensive benchmark with an offline EMS and two online strategies is performed under two driving cycles. In comparison with the online strategies, the proposed method based on RL reduces the defined trip cost up to 11.5 % and 13.08 % under the Real driving cycle while having a higher cost than the offline strategy by 4.78 %.

Automated summary

This paper proposes a three-layer online EMS for efficient power distribution in a multi-stack fuel cell hybrid electric vehicle. The method continuously updates the characteristics of each fuel cell and battery using online estimators, and uses a rule-based approach to improve the calculation speed. The power distribution is achieved using a Q-learning algorithm based on reinforcement learning. The proposed method significantly reduces the trip cost compared to other online strategies and has a slightly higher cost compared to the offline strategy.