Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning

Efficient and flexible energy management strategy can play an important role in energy conservation in building sector. The model-free reinforcement learning control of building energy systems generally requires an enormous amount of training data and low learning efficiency creates an obstacle to practice. This work proposes a hybrid model-based reinforcement learning framework to optimize the indoor thermal comfort and energy cost-saving performances of a ZEH (zero energy house) space heating system using relatively short-period monitored data. The reward function is designed regarding energy cost, PV self-consumption and thermal discomfort, proposed agents can interact with the reduced-order thermodynamic model and an uncertain environment, and makes optimal control policies through the learning process. Simulation results demonstrate that proposed agents achieve efficient convergence, D3QN presents a superiority of convergence performance. To evaluate the per-formances of proposed algorithms, the trained agents are tested using monitored data. With learned policies, the self-learning agents could balance the needs of thermal comfort, energy cost saving and increasing on-site PV consumption compared with the baselines. The comparative analysis shows that D3QN achieved over 30% cost savings compared with measurement results. D3QN outperforms DQN and Double DQN agents in test scenarios maintaining more stable temperatures under various outside conditions.

Automated summary

An efficient and flexible energy management strategy is crucial for energy conservation in the building sector. This study proposes a hybrid model-based reinforcement learning framework that uses short-term monitored data to optimize indoor thermal comfort and energy cost-saving performance. Simulation results demonstrate the efficiency and superiority of the proposed framework, with the D3QN agent achieving over 30% cost savings compared to measurement results.