Optimal Charging Control of Energy Storage Systems for Pulse Power Load Using Deep Reinforcement Learning in Shipboard Integrated Power Systems

In this article, the charging control of the energy storage system for the pulse power load accommodation in a shipboard integrated power system (SIPS) is formulated as an optimal control problem. The SIPS is an input-affine nonlinear system with randomness and fast dynamics. The improved twin-delayed deep deterministic policy gradient algorithm -one of the deep reinforcement learning (DRL) algorithms, is proposed to solve this optimal control problem. The proposed DRL-based control solution considers the issues regarding the reward function design and input and ramp rate constraints handling for control variables. The proposed approach linked the optimal control and DRL framework. Test cases demonstrated that we could utilize DRL algorithms to control the nonlinear system with fast dynamics by following the specific reward function design, data sampling, and constraints handling techniques.

Automated summary

In this article, the charging control of the energy storage system for pulse power load accommodation in a shipboard integrated power system is studied. An improved twin-delayed deep deterministic policy gradient algorithm is proposed to solve this control problem. The proposed method addresses issues such as reward function design and constraint handling for control variables.