Mott physics on helical edges of two-dimensional topological insulators

We study roles of electron correlations on topological insulators or quantum spin Hall insulators on honeycomb lattice with spin-orbit interaction. Accurate variationalMonteCarlo calculations with a large number of variational parameters show that the increasing on-site Coulomb interactions cause a strong suppression of the charge Drude weight in the helical-edge metallic states leading to a presumable Mott transition (or strong crossover) from a conventional topological insulator to an edge Mott insulator before a transition to a bulk antiferromagnetic insulator. The intermediate bulk-topological and edge-Mott-insulator phase has a helical spin-liquid character with time-reversal symmetry.