Exploring the potential of time-series transformers for process modeling and control in chemical systems: An inevitable paradigm shift?

The last two years have seen groundbreaking advances in natural language processing (NLP) with the advent of applications like ChatGPT, Codex, and ChatSonic. This revolution is powered by the development of cutting-edge transformer models that leverage multiheaded attention mechanisms, positional encoding, and highly efficient transfer learning capabilities. Despite these remarkable advances, there is still work to be done to fully realize the practical applicability of transformers in chemical systems. Thus, we are excited to present our latest work, which highlights the immense potential of transformers for non-trivial multivariate time-series prediction tasks with high-value implications in process monitoring, control, and optimization. Specifically, impressive prediction capabilities of first-generation time-series transformers (TSTs) were demonstrated by developing, testing, and comparing TSTs with existing models. Further, the practical applicability of TSTs was highlighted by developing a first-of-a-kind TST-based model predictive controller (MPC). More importantly, the current work provides a concrete foundation for exploring promising new directions, such as the development of largescale TSTs leveraging transfer learning for modeling of new process equipment, and plant-level multisource aggregative cognitive models for fault prognosis and prevention. We are excited to see what the future holds as we continue to push the boundaries of what is possible with these 'transformer-tive' technologies.(c) 2023 Institution of Chemical Engineers. Published by Elsevier Ltd. All rights reserved.

Automated summary

The past two years have witnessed groundbreaking advances in natural language processing, driven by the development of cutting-edge transformer models. Despite remarkable progress, there is still work to be done to fully apply transformers in chemical systems.