Evidence of the Berezinskii-Kosterlitz-Thouless phase in a frustrated magnet

The Berezinskii-Kosterlitz-Thouless (BKT) mechanism, building upon proliferation of topological defects in 2D systems, is the first example of phase transition beyond the Landau-Ginzburg paradigm of symmetry breaking. Such a topological phase transition has long been sought yet undiscovered directly in magnetic materials. Here, we pin down two transitions that bound a BKT phase in an ideal 2D frustrated magnet TmMgGaO4, via nuclear magnetic resonance under in-plane magnetic fields, which do not disturb the low-energy electronic states and allow BKT fluctuations to be detected sensitively. Moreover, by applying out-of-plane fields, we find a critical scaling behavior of the magnetic susceptibility expected for the BKT transition. The experimental findings can be explained by quantum Monte Carlo simulations applied on an accurate triangular-lattice Ising model of the compound which hosts a BKT phase. These results provide a concrete example for the BKT phase and offer an ideal platform for future investigations on the BKT physics in magnetic materials. The Berezinskii-Kosterlitz-Thouless (BKT) phase transition has long been sought yet undiscovered directly in magnetic materials. Here, the authors identify two phase transitions with BKT fluctuations detected by NMR and critical scaling behavior in magnetic susceptibility expected for the BKT transition in a frustrated magnet TmMgGaO4.