Joint optimization of maintenance and quality inspection for manufacturing networks based on deep reinforcement learning

Most existing studies on joint optimization of manufacturing systems (MS) focus on small-scale systems with simple structures, such as the single-machine, simple serial, or parallel MS. Simultaneously, traditional algorithms utilized in small-scale MS always show an insufficiency in solving large-scale dynamic MS with complex structures, such as manufacturing networks. Therefore, considering the effectiveness of reinforcement learning on the infinite-horizon Markov Decision Process (MDP), this paper presents a joint optimization problem of preventive maintenance and work-in-process quality inspection for manufacturing networks with reliabilityquality interactions. First, dynamic reliability and quality models are proposed at the machine level to cope with complex interactions in manufacturing networks. Second, based on the MDP-based optimization model, the proposed Deep Deterministic Policy Gradient (DDPG) algorithm realizes the optimal reliability-quality joint control in manufacturing networks. Besides, it also offers a novel mixed action space containing discrete maintenance and continuous quality inspection, which could satisfy the action diversity in actual production. At last, training and experiments imply our algorithm is more adaptable to diverse manufacturing scenarios than traditional ones. Also, it is proved that more-frequent state observations for learning cannot help the constructed reinforcement learning model get a better control policy because of the information redundancy.

Automated summary

This study proposes a reinforcement learning-based method to solve the joint optimization problem of preventive maintenance and work-in-process quality inspection in manufacturing networks. By introducing dynamic reliability and quality models, and employing the Deep Deterministic Policy Gradient algorithm, the optimal joint control of reliability and quality in manufacturing networks is achieved.