A new framework integrating reinforcement learning, a rule-based expert system, and decision tree analysis to improve building energy flexibility

This study presents a new framework that integrates machine learning and a domain knowledgebased expert system to improve building energy flexibility. In this framework, a rule-based expert system was used to maximize the self-consumption of solar photovoltaics (PV) power, while a reinforcement learning (RL) agent was constructed to efficiently optimize the grid power import for battery charging and facilitate decision-making for battery discharging in response to the time of use electricity prices. Meanwhile, a Classification and Regression Tree (CART) model was developed to quantitatively analyze the relationships between building energy flexibility and external variables of interest to enhance the explainability of the framework. This work integrates the advantages of safety and simpleness of the domain knowledge-based expert system and the exploration and optimization capability of RL into building energy management. The performance of the proposed framework was evaluated using the four-year data collected from a real net zero energy office building. The system cost reduction ratio, self-consumption ratio, and selfsufficiency ratio were used as the energy flexibility indicators. The results showed that the integration of the RL and the rule-based expert system was able to reduce the electricity cost and grid power consumption by 7.0% and 10.6% respectively, and increase the self-consumption of PV power by 9.2% as compared with the use of the rule-based expert system only. The CART analysis also showed that external conditions can significantly influence the level of building energy flexibility. For instance, the average daily cost reduction ratio was 0.89 out of 1.0 when the daily maximum solar irradiance was above 717.5 W/m2, while it decreased to 0.28 when the daily mean solar irradiance was below 62.4 W/m2. This strategy can be used to facilitate building demand-side management and improve the design and control of building energy systems for enhanced demand flexibility.

Automated summary

This study proposes a new framework that combines machine learning and a domain knowledge-based expert system to enhance building energy flexibility. The framework utilizes a rule-based expert system to optimize self-consumption of solar PV power, while employing a reinforcement learning agent to optimize grid power import for battery charging and decision-making for battery discharging. Additionally, a Classification and Regression Tree model is developed to analyze the relationships between building energy flexibility and external variables. This framework integrates the benefits of safety and simplicity from the expert system and the exploration and optimization capabilities of RL for building energy management. Overall, this study obtains a score of 9 out of 10 for its significance.