Visual interpretation of deep deterministic policy gradient models for energy consumption prediction

Accurate prediction of energy consumption is pivotal to achieving sustainable building energy objectives, and Deep Reinforcement Learning (DRL) has demonstrated efficacy in this regard. Nevertheless, efficient model training within DRL remains challenging for practitioners due to the need for expertise in Reinforcement Learning (RL) and parameter tuning. Moreover, the invisible mechanism of DRL models raises doubts among users, impeding subsequent tasks. To address these challenges, a visual analytics system named DDPGVis is proposed in this work, which focuses on exploring the experience data generated by Deep Deterministic Policy Gradient (DDPG) models used for energy consumption prediction. Specifically, temporal aggregation of steps is employed to heighten the efficiency of subsequent analysis. Feature importance analysis and dimensionality reduction of state data are utilized to help users understand the high-dimensional environment space. Simultaneously, experience data is subjected to spatio-temporal modeling, yielding dynamic network diagrams, which are utilized to analyze the experience correlations. Except for showcasing the statistics and results from the analysis of state and experience data, DDPGVis also provides a recommendation view for assisting users in parameter tuning. In corporation with three non-reinforcement learning experts, case studies demonstrate that DDPGVis can help users understand the model training process, diagnose model anomalies, and optimize the model efficiency. Compared to the parameters initially set by the same expert for energy consumption prediction, DDPGVis can recommend a better configuration that contributes to a reduction of MAE, MAPE, and RMSE by 41%, 55.62%, and 28.03%, respectively, and an increase of R2 by 7.42%.

Automated summary

This paper introduces a visual analytics system called DDPGVis, which explores the experience data generated by Deep Deterministic Policy Gradient (DDPG) models for energy consumption prediction. The system uses temporal aggregation and feature importance analysis to help users understand the high-dimensional environment space. DDPGVis also provides a recommendation view for parameter tuning. Experimental results show that DDPGVis can help users understand the model training process and provide optimization recommendations.