Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control

Control of batch processes is a difficult task due to their complex nonlinear dynamics and unsteady-state operating conditions within batch and batch-to-batch. It is expected that some of these challenges can be addressed by developing control strategies that directly interact with the process and learning from experiences. Recent studies in the literature have indicated the advantage of having an ensemble of ac-tors in actor-critic Reinforcement Learning (RL) frameworks for improving the policy. The present study proposes an actor-critic RL algorithm, namely, twin actor twin delayed deep deterministic policy gradi-ent (TATD3), by incorporating twin actor networks in the existing twin-delayed deep deterministic policy gradient (TD3) algorithm for the continuous control. In addition, two types of novel reward functions are also proposed for TATD3 controller. We showcase the efficacy of the TATD3 based controller for var-ious batch process examples by comparing it with some of the existing RL algorithms presented in the literature. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

Control of batch processes is a challenging task due to their complex dynamics and non-steady state operating conditions. Developing control strategies that directly interact with the process and learning from experiences can help address some of these challenges. The study introduces a novel actor-critic RL algorithm and demonstrates its efficacy in various batch process examples.