A bi-level reinforcement learning model for optimal scheduling and planning of battery energy storage considering uncertainty in the energy-sharing community

Sharing of battery energy storage systems (BESS) in the energy community by reflecting the real world can play a significant role in achieving carbon neutrality. Therefore, this study aimed to develop a bi-level reinforcement learning (RL) model of BESS considering uncertainty in the energy-sharing community for the following optimization strategies: (i) short-term scheduling model for optimal electricity flows considering operational objectives (i.e., self-sufficiency rate (SSR), peak load, and economic profit); and (ii) long-term planning model for optimal BESS plan (i.e., install, replace, and disuse) along with battery types (new or reused batteries). A case study in the South Korea Nonhyeon neighborhood was conducted to evaluate the developed bi-level RL model feasibility based on future scenarios considering the time-dependent variables. The developed model increased economic profit by up to 18,830 USD compared to the rule-based model. Compared to the case where BESS was not installed, SSR increased by up to 7.79% and peak demand decreased by up to 1.31 kWh. These results show that the developed model could maximize the economic feasibility of community-shared BESS by reflecting the uncertainty in the real world, ultimately benefiting participants in the energy-sharing community.

Automated summary

This study aimed to develop a bi-level reinforcement learning model for battery energy storage systems in the energy-sharing community, considering uncertainty. The model includes short-term scheduling for optimal electricity flows and long-term planning for optimal BESS plans. A case study in South Korea showed that the developed model increased economic profit, self-sufficiency rate, and decreased peak demand.