Model-Free Emergency Frequency Control Based on Reinforcement Learning

Unexpected large power surges will cause instantaneous grid shock and, thus, emergency control plans must be implemented to prevent the system from collapsing. In this article, with the aid of reinforcement learning, novel model-free control (MFC)-based emergency control schemes are presented. First, multi-Q-learning-based emergency plans are designed for limited emergency scenarios by using offline-training-online-approximation methods. To solve the more general multiscenario emergency control problem, a deep deterministic policy gradient (DDPG) algorithm is adopted to learn near-optimal solutions. With the aid of deep Q network, DDPG-based strategies have better generalization abilities for unknown and untrained emergency scenarios, and thus are suitable for multiscenario learning. Through simulations using benchmark systems, the proposed schemes are proven to achieve satisfactory performances.

Automated summary

This article introduces novel emergency control schemes using reinforcement learning, including multi-Q learning and deep deterministic policy gradient algorithms. These strategies are designed for various emergency scenarios and have shown satisfactory performance in simulations using benchmark systems. The techniques demonstrate better generalization abilities for unknown and untrained emergency scenarios, making them suitable for multiscenario learning.