Interaction-driven quantum phase transitions in fractional topological insulators

We study two species of (or spin-1/2) fermions with short-range intraspecies repulsion in the presence of opposite (effective) magnetic field, each at Landau level filling factor 1/3. In the absence of interspecies interaction, the ground state is simply two copies of the 1/3 Laughlin state, with opposite chirality, representing the fractional topological insulator (FTI) phase. We show this phase is stable against moderate interspecies interactions. However, strong enough interspecies repulsion leads to phase separation, while strong enough interspecies attraction drives the system into a superfluid phase. We obtain the phase diagram through exact diagonalization calculations. The FTI-superfluid phase transition is shown to be in the (2 + 1)-dimensional XY universality class, using an appropriate Chern-Simons-Ginsburg-Landau effective field theory.