A Fast Technique for Smart Home Management: ADP With Temporal Difference Learning

This paper presents a computationally efficient smart home energy management system (SHEMS) using an approximate dynamic programming (ADP) approach with temporal difference learning for scheduling distributed energy resources. This approach improves the performance of an SHEMS by incorporating stochastic energy consumption and PV generation models over a horizon of several days, using only the computational power of existing smart meters. In this paper, we consider a PV-storage (thermal and battery) system, however, our method can extend to multiple controllable devices without the exponential growth in computation that other methods such as dynamic programming (DP) and stochastic mixed-integer linear programming (MILP) suffer from. Specifically, probability distributions associated with the PV output and demand are kernel estimated from empirical data collected during the Smart Grid Smart City project in NSW, Australia. Our results show that ADP computes a solution much faster than both DP and stochastic MILP, and provides only a slight reduction in quality compared to the optimal DP solution. In addition, incorporating a thermal energy storage unit using the proposed ADP-based SHEMS reduces the daily electricity cost by up to 263% without a noticeable increase in the computational burden. Moreover, ADP with a two-day decision horizon reduces the average yearly electricity cost by a 4.6% over a daily DP method, yet requires less than half of the computational effort.