Light-shift-induced quantum phase transitions of a Bose-Einstein condensate in an optical cavity

In this paper we reveal the rich ground-state properties induced by the strong nonlinear atom-photon interaction which has been found in the recent experiment about a Bose-Einstein condensate coupling with a high-finesse cavity [Nature (London) 464, 1301 (2010)]. Two detuning-dependent phase diagrams are revealed by investigating the experimentally measurable atomic population. In particular, two quantum phase transitions from the superradiant phase or the normal phase to a dynamically unstable phase are predicted in the blue detuning. Moreover, the three-phase coexistence points are found. It is also demonstrated that these predicted quantum phase transitions are the intrinsic transitions governed only by the second-order derivative of the ground-state energy. Finally, we also point out that the region involving coexistence of the superradiant phase and the normal phase previously predicted cannot happen in the ground state.