Predictive Maintenance Model for IIoT-Based Manufacturing: A Transferable Deep Reinforcement Learning Approach

The Industrial Internet of Things (IIoT) is crucial for accurately assessing the state of complex equipment in order to perform predictive maintenance (PdM) successfully. However, existing IIoT-based PdM frameworks do not consider the influence of various practical yet complex system factors, such as the real-time production states, machine health, and maintenance manpower resources. For this reason, we propose a generic PdM optimization framework to assist maintenance teams in prioritizing and resolving maintenance task conflicts under real-world manufacturing conditions. Specifically, the PdM framework aims to jointly optimize the edge-based machine network uptime and the allocation of manpower resources in a stochastic IIoT-enabled manufacturing environment using the model-free deep reinforcement learning (DRL) methods. Since DRL requires a significant amount of training data, we propose and demonstrate the use of the transfer learning (TL) method to assist DRL in learning more efficiently by incorporating expert demonstrations, termed TL with demonstrations (TLDs). TLD reduces training wall time by 58% compared to baseline methods, and we conduct numerous experiments to illustrate the performance, robustness, and scalability of TLD. Finally, we discuss the general benefits and limitations of the proposed TL method, which are not well addressed in the existing literature but could be beneficial to both researchers and industry practitioners.

Automated summary

The Industrial Internet of Things (IIoT) is crucial for successful predictive maintenance (PdM). However, existing PdM frameworks do not consider practical system factors. Therefore, we propose a generic PdM optimization framework using deep reinforcement learning (DRL) to optimize machine network uptime and resource allocation. We also propose transfer learning (TL) to improve training efficiency and reduce training time.