Microgrid energy management using deep Q-network reinforcement learning

This paper proposes a deep reinforcement learning-based approach to optimally manage the different energy resources within a microgrid. The proposed methodology considers the stochas-tic behavior of the main elements, which include load profile, generation profile, and pricing signals. The energy management problem is formulated as a finite horizon Markov Decision Process (MDP) by defining the state, action, reward, and objective functions, without prior knowledge of the tran-sition probabilities. Such formulation does not require explicit model of the microgrid, making use of the accumulated data and interaction with the microgrid to derive the optimal policy. An efficient reinforcement learning algorithm based on deep Q-networks is implemented to solve the developed formulation. To confirm the effectiveness of such methodology, a case study based on a real micro-grid is implemented. The results of the proposed methodology demonstrate its capability to obtain online scheduling of various energy resources within a microgrid with optimal cost-effective actions under stochastic conditions. The achieved costs of operation are within 2% of those obtained in the optimal schedule.(c) 2022 THE AUTHORS. Published by Elsevier BV on behalf of Faculty of Engineering, Alexandria University. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/ 4.0/).

Automated summary

This paper proposes a deep reinforcement learning-based approach to optimally manage different energy resources within a microgrid. The proposed methodology considers the stochastic behavior of the main elements and formulates the energy management problem as a finite horizon Markov Decision Process (MDP) without prior knowledge of transition probabilities. An efficient reinforcement learning algorithm based on deep Q-networks is implemented to solve the problem. The results of a case study based on a real microgrid demonstrate the effectiveness of the proposed methodology in achieving optimal cost-effective scheduling of energy resources under stochastic conditions.