Multi-period, multi-timescale stochastic optimization model for simultaneous capacity investment and energy management decisions for hybrid Micro-Grids with green hydrogen production under uncertainty

Given the steep rises in renewable energy's proportion in the global energy mix expected over several decades, a systematic way to plan the long-term deployment is needed. The main challenges are how to handle the sig-nificant uncertainties in technologies and market dynamics over a large time horizon. The problem is further complicated by the fast-timescale volatility of renewable energy sources, potentially causing grid instability and unfulfilled demands. As a remedy, energy storage and power-to-hydrogen systems are considered in conjunction with energy management system but doing so raises the complexity of the planning problem further. In this work, the long-term capacity planning for a hybrid microgrid (HM) system is formulated as a multi-period stochastic decision problem that considers uncertainties occurring at multiple timescales. Long-term capacity decisions are inherently linked with energy dispatch and storage decisions occurring at fast-timescale and it is best to solve for them simultaneously. However, the computational demand for solving it becomes quickly intractable with problem size. To this end, we propose to develop a Markov decision process (MDP) formulation of the problem and use simulation-based reinforcement learning for multi-period capacity investments of the planning horizon. The MDP includes the policies used for dispatch and storage operation, which are represented by linear programming as a part of the simulation model. The effectiveness of our proposed method is demonstrated with a case study, where decisions over multiple decades are considered along with various un-certainties of multi-timescales. Economic and environmental assessments are performed, providing valuable guidelines for government's energy policy.

Automated summary

This article discusses the importance of renewable energy in the global energy mix and the challenges of long-term deployment planning. To address uncertainties in technologies and market dynamics, a solution using energy storage and power-to-hydrogen systems in conjunction with energy management systems is proposed. We also demonstrate the applicability of Markov decision processes and simulation-based reinforcement learning for multi-period capacity investments.