A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing

In deregulated environment, the wind power producers (WPPs) will face the challenge of how to increase their revenues under uncertainties of wind generation and electricity price. This paper proposes a method based on deep reinforcement learning (DRL) to address this issue. A data-driven controller that directly maps the input observations, i.e., the forecasted wind generation and electricity price, to the control actions of the wind farm, i.e., the charge/discharge schedule of the relevant energy storage system (ESS) and the reserve purchase schedule, is trained according to the method. By the well-trained controller, the influence of the uncertainties of wind power and electricity price on the revenue can be automatically involved and an expected optimal decision can be obtained. Furthermore, a targeted DRL algorithm, i.e., the Rainbow algorithm, is implemented to improve the effectiveness of the controller. Especially, the algorithm can overcome the limitation of the conventional reinforcement learning algorithms that the input states must be discrete, and thus the validity of the control strategy can be significantly improved. Simulation results illustrate that the proposed method can effectively cope with the uncertainties and bring high revenues to the WPPs.