Putting competing orders in their place near the Mott transition

We describe the localization transition of superfluids on two-dimensional lattices into commensurate Mott insulators with average particle density p/q (p,q relatively prime integers) per lattice site. For bosons on the square lattice, we argue that the superfluid has at least q degenerate species of vortices which transform under a projective representation of the square-lattice space group (a PSG). The formation of a single-vortex condensate produces the Mott insulator, which is required by the PSG to have density wave order at wavelengths of q/n lattice sites (n integer) along the principle axes; such a second-order transition is forbidden in the Landau-Ginzburg-Wilson frame-work. We also discuss the superfluid-insulator transition in the direct boson representation and find that an interpretation of the quantum criticality in terms of deconfined fractionalized bosons is only permitted at special values of q for which a permutative representation of the PSG exists. We argue [and demonstrate in detail in a companion paper: L. Balents , following paper, Phys. Rev. B 71, 144509 (2005)] that our results apply essentially unchanged to electronic systems with short-range pairing, with the PSG determined by the particle density of Cooper pairs. We also describe the effect of static impurities in the superfluid: the impurities locally break the degeneracy between the q vortex species, and this induces density-wave order near each vortex. We suggest that such a theory offers an appealing rationale for the local density-of-states modulations observed by Hoffman [Science 295, 466 (2002)], in scanning tunneling microscopy (STM) studies of the vortex lattice of Bi2Sr2CaCu2O8+delta and allows a unified description of the nucleation of density-wave order in zero and finite magnetic fields. We note signatures of our theory that may be tested by future STM experiments.