A Novel Automatic Generation Control Method Based on the Large-Scale Electric Vehicles and Wind Power Integration Into the Grid

In order to solve the problem of frequency instability of power system due to strong random disturbance caused by large-scale electric vehicles and wind power grid connection, an improved reinforcement learning algorithm, namely, optimistic initialized double Q, is proposed in this article from the perspective of automatic generation control. The proposed algorithm uses the optimistic initialization principle to expand the agent action exploration space, so as to prevent Q-learning from falling into local optimum by greedy strategy; meanwhile, it integrates double Q-learning to solve the problem of over-estimation of action value in traditional reinforcement learning based on Q-learning. In the algorithm, the hyperparameter alpha(tau) is introduced to improve the learning efficiency, and the reward b(tau) based on exploration times is introduced to increase the Q value estimation to drive the exploration of the algorithm, so as to obtain the optimal solution. By simulating the twoarea load frequency control model integrated with large-scale electric vehicles and the four-area interconnected power grid model integrated with large-scale wind power generation, it is verified that the proposed algorithm can obtain the global optimal solution, thus effectively solvinng the frequency instability caused by strong random disturbance in the grid-connected mode of large-scale wind power generation, and compared with many reinforcement learning algorithms, the proposed algorithm has better control performance.

Automated summary

This article proposes an improved reinforcement learning algorithm to solve the problem of frequency instability in power systems caused by large-scale electric vehicles and wind power grid connection. The algorithm expands the exploration space using an optimistic initialization principle and integrates double Q-learning to address the over-estimation issue. Simulation results demonstrate that the proposed algorithm obtains the global optimal solution and outperforms other reinforcement learning algorithms in terms of control performance.