Coordinated control of gas supply system in PEMFC based on multi-agent deep reinforcement learning

In proton exchange membrane fuel cells (PEMFCs), the hydrogen supply system and air supply system jointly impact the output characteristics, and there is a coordination problem between these two systems. To solve this coordination problem, an intelligent control framework is presented that considers the coordination between the air flux controller and hydrogen flux controller in PEMFCs, and an ensemble imitation learning multi-trick deep deterministic policy gradient (EILMMA-DDPG) is advanced for this framework. The algorithm proposed here complies with an ensemble imitation learning policy, i.e., exploiting multiple reinforcement learning explorers that contain actor networks to perform distributed exploration in the environment, thereby improving the exploration efficiency. Moreover, a control algorithm explorer that contains various conventional control algorithms is presented to create model samples over a range of scenarios in an attempt to address sparse rewards and improve the training efficiency in conjunction with an experience probability replay mechanism. Next, multiple tricks are adopted to improve the overestimated Q value. Finally, a model-free intelligent control algorithm capable of coordinating controllers and exhibiting a better global searching ability is developed. In addition, the proposed algorithm is adopted in the control framework of the air and hydrogen supply system in PEMFCs. Furthermore, as revealed from the simulated results, the proposed intelligent control framework can more effectively control the oxygen excess rate (OER) and output voltage. (c) 2021 Hydrogen Energy Publications LLC. Published by Elsevier Ltd. All rights reserved.

Automated summary

An intelligent control framework is proposed for coordinating the air and hydrogen supply systems in PEMFCs, using ensemble imitation learning and multi-trick deep deterministic policy gradient approach to enhance exploration efficiency. Multiple reinforcement learning explorers and control algorithm explorers are utilized to address sparse rewards and improve training efficiency. Multiple tricks are applied to improve the overestimated Q value, resulting in a model-free intelligent control algorithm with better global searching ability.