Energy Management of PV-Storage Systems: Policy Approximations Using Machine Learning

In this paper, we propose a policy function approximation (PFA) algorithm using machine learning to effectively control photovoltaic (PV)-storage systems. The algorithm uses an offline policy planning stage and an online policy execution stage. In the planning stage, a suitable machine learning technique is used to generate models that map states (inputs) and decisions (outputs) using training data. The training dataset is generated by solving a deterministic smart home energy management problem using a suitable optimization technique [e.g., mathematical programming or dynamic programming (DP)]. In the execution stage, the model generated by the machine learning algorithm is then used to generate fast real-time decisions. Since the decisions can be made in real-time, the policy can rely on up-to-date information on PV output, electrical demand, and battery state of charge. Moreover, we can use PFA models over a long period of time (i.e., months) without having to update them but still obtain similar quality solutions. Our results show that the solutions from the PFAs are close to the best solutions obtained using DP and approximate DP, which have the drawback of requiring an optimization problem to be solved before the beginning of each day or as new information on demand or PV becomes available.