Holographic endpoint of spatially modulated phase transition

In a previous paper [S. Nakamura, H. Ooguri, and C. S. Park, Phys. Rev. D 81, 044018 (2010)], we showed that the Reissner-Nordstrom black hole in the five-dimensional anti-de Sitter space coupled to the Maxwell theory with the Chern-Simons term is unstable when the Chern-Simons coupling is sufficiently large. In the dual conformal field theory, the instability suggests a spatially modulated phase transition. In this paper, we construct and analyze nonlinear solutions which describe the endpoint of this phase transition. In the limit where the Chern-Simons coupling is large, we find that the phase transition is of the second order with the mean field critical exponent. However, the dispersion relation with the Van Hove singularity enhances quantum corrections in the bulk, and we argue that this changes the order of the phase transition from the second to the first. We compute linear response functions in the nonlinear solution and find an infinite off-diagonal DC conductivity in the new phase.