Reduced-order modeling of fluid flows with transformers

Reduced-order modeling (ROM) of fluid flows has been an active area of research for several decades. The huge computational cost of direct numerical simulations has motivated researchers to develop more efficient alternative methods, such as ROMs and other surrogate models. Similar to many application areas, such as computer vision and language modeling, machine learning and data-driven methods have played an important role in the development of novel models for fluid dynamics. The transformer is one of the state-of-the-art deep learning architectures that has made several breakthroughs in many application areas of artificial intelligence in recent years, including but not limited to natural language processing, image processing, and video processing. In this work, we investigate the capability of this architecture in learning the dynamics of fluid flows in a ROM framework. We use a convolutional autoencoder as a dimensionality reduction mechanism and train a transformer model to learn the system's dynamics in the encoded state space. The model shows competitive results even for turbulent datasets.

Automated summary

Reduced-order modeling (ROM) is an active area of research in fluid flows, where more efficient alternative methods like ROMs and surrogate models have been developed to overcome the high computational cost of direct numerical simulations. Machine learning and data-driven methods, similar to other application areas, have played an important role in the development of novel models for fluid dynamics. In this study, the capability of the state-of-the-art deep learning architecture, transformer, is investigated for learning the dynamics of fluid flows in a ROM framework. A convolutional autoencoder is used for dimensionality reduction, and a transformer model is trained to learn the system's dynamics in the encoded state space. The model shows competitive results even for turbulent datasets.