Deep Reinforcement Learning-Based Optimal Control of DC Shipboard Power Systems for Pulsed Power Load Accommodation

To accommodate the pulsed power load (PPL) in the dc shipboard power system, the charging performance of the energy storage system (ESS) specialized for the PPL needs to be guaranteed, which leads to a challenging power system optimal control problem due to multiple objectives, operational constraints, complex nonlinear system structure, and uncertainties. This paper addresses this problem by using a model-free optimal control method based on the deep reinforcement learning (DRL). First, a dc shipboard power system optimal control problem with three control objectives and the input constraints is formulated, where three objectives include the fast ESS charge, the dc bus voltage regulation, and the proportional load current sharing. Then, to solve this problem, a DRL control framework based on the improved twin-delayed deep deterministic policy gradient (TD3) algorithm is developed, which adopts a modified critic network predicting technique and a stack-based data sampling strategy that are suitable for this fast-dynamic power system. The proposed method links the DRL framework with the optimal control. With the reward function being properly designed, the presented DRL control can well realize three control objectives. Case studies considering various operating conditions of the power system verify its effectiveness.

Automated summary

This paper addresses the challenging power system optimal control problem in the dc shipboard power system by using a model-free optimal control method based on deep reinforcement learning (DRL). A DRL control framework based on the improved twin-delayed deep deterministic policy gradient (TD3) algorithm is developed to solve the dc shipboard power system optimal control problem with three control objectives and input constraints, which includes the fast ESS charge, the dc bus voltage regulation, and the proportional load current sharing. The proposed method effectively links the DRL framework with optimal control.