Learning-Based Demand Response for Privacy-Preserving Users

Demand response (DR), as a vital component of smart grid, plays an important role in shaping the load profiles in order to improve system reliability and efficiency. Incentive-based DR has been used in many DR programs by incentivizing customers to adapt their loads to supply availability. Note that users' behavior patterns can be easily identified from fine-grained power consumption when interacting with the load serving entity (LSE), giving rise to serious privacy concerns. One common approach to address the privacy threats is to incorporate perturbations in users' load measurements. Although it can protect the users' privacy, yet the usage data modification would degrade the LSE's performance in achieving an optimal incentive strategy due to unknown characteristics of the augmented perturbations. In this paper, we cast the incentive-based DR problem as a stochastic Stackelberg game. To tackle the challenge induced by users' privacy protection behaviors, we propose a two-timescale reinforcement learning algorithm to learn the optimal incentive strategy under users' perturbed responses. The proposed algorithm computes the expected utility cost to mitigate the impacts of the random characteristics of the augmented perturbations and then updates the incentive strategy based on the perceived expected utility costs. We derive the conditions under which the proposed incentive scheme converges almost surely to an epsilon-optimal strategy. The efficacy of the proposed algorithm is demonstrated using extensive numerical simulation using real data.