A model-based deep reinforcement learning approach to the nonblocking coordination of modular supervisors of discrete event systems

Modular supervisory control may lead to conflicts among the modular supervisors for large-scale discrete event systems. The existing methods for ensuring nonblocking control of modular supervisors either exploit favorable structures in the system model to guarantee the nonblocking property of modular supervisors or employ hierarchical model abstraction methods for reducing the computational complexity of designing a nonblocking coordinator. The nonblocking modular control problem is, in general, NP-hard. This study integrates supervisory control theory and a model-based deep reinforcement learning method to synthesize a nonblocking coordinator for the modular supervisors. The deep reinforcement learning method significantly reduces the computational complexity by avoiding the computation of synchronization of multiple modular supervisors and the plant models. The supervisory control function is approximated by the deep neural network instead of a large-sized finite automaton. Furthermore, the proposed model-based deep reinforcement learning method is more efficient than the standard deep Q network algorithm.

Automated summary

Modular supervisory control may cause conflicts among supervisors in large-scale discrete event systems. Existing methods for nonblocking control either utilize favorable system structures or adopt hierarchical model abstraction methods to reduce computational complexity. This study integrates supervisory control theory with model-based deep reinforcement learning to synthesize a nonblocking coordinator. The proposed method significantly reduces complexity by avoiding synchronization computation and approximating the control function using a deep neural network.