Transformer Variational Wave Functions for Frustrated Quantum Spin Systems

The transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the vision transformer (ViT) architecture. The key feature is the ability to describe long-range correlations among the elements of the input sequences, through the so-called self-attention mechanism. Here, we propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states for quantum many-body systems, the ViT wave function. We apply this idea to the one-dimensional J1-J2 Heisenberg model, demonstrating that a relatively simple parametrization gets excellent results for both gapped and gapless phases. In this case, excellent accuracies are obtained by a relatively shallow architecture, with a single layer of self-attention, thus largely simplifying the original architecture. Still, the optimization of a deeper structure is possible and can be used for more challenging models, most notably highly frustrated systems in two dimensions. The success of the ViT wave function relies on mixing both local and global operations, thus enabling the study of large systems with high accuracy.

Automated summary

The ViT architecture, which utilizes the self-attention mechanism to capture long-range correlations, has become the leading model for natural language processing and computer vision tasks. We propose an adaptation of this architecture, called ViT wave function, with complex parameters for variational neural-network states of quantum many-body systems. Applying this idea to the one-dimensional J1-J2 Heisenberg model, we demonstrate that a relatively simple parametrization achieves excellent results for both gapped and gapless phases.