Exploring global attention mechanism on fault detection and diagnosis for complex engineering processes

Considering about slow drift and complicated relationships among process variables caused by corrosion, fatigue, and so on in complex chemical engineering processes, an Industrial Process Optimization ViT (IPO-ViT) method was proposed to explore the global receptive field provided by self-attention mechanism of Vision Transformer (ViT) on fault detection and diagnosis (FDD). The applications on data sampled from both a real industrial process and the Tennessee Eastman (TE) process showed superior performance of the global attention-based method (IPO-ViT) over other typical local receptive fields deep learning methods without increasing sample and computation requirements. Moreover, results on six different variants in combing local, shallow filtering and global receptive field mechanisms unravel that the local attention explosion, the information alignment, and the expression capability are three major challenges for further improving on industrial applications of complex deep learning network structures.

Automated summary

Considering the slow drift and complicated relationships caused by corrosion and fatigue in complex chemical engineering processes, a method called Industrial Process Optimization ViT (IPO-ViT) was proposed. The method makes use of the self-attention mechanism of Vision Transformer (ViT) to explore the global receptive field for fault detection and diagnosis (FDD). The results from real industrial process data and Tennessee Eastman (TE) process data showed that IPO-ViT outperforms other deep learning methods with local receptive fields, without the need for additional samples or computations. Additionally, the study identified the challenges of local attention explosion, information alignment, and expression capability in improving complex deep learning network structures for industrial applications.