Negative-Mass Instability of the Spin and Motion of an Atomic Gas Driven by Optical Cavity Backaction

We realize a spin-orbit interaction between the collective spin precession and center-of-mass motion of a trapped ultracold atomic gas, mediated by spin-and position-dependent dispersive coupling to a driven optical cavity. The collective spin, precessing near its highest-energy state in an applied magnetic field, can be approximated as a negative-mass harmonic oscillator. When the Larmor precession and mechanical motion are nearly resonant, cavity mediated coupling leads to a negative-mass instability, driving exponential growth of a correlated mode of the hybrid system. We observe this growth imprinted on modulations of the cavity Held and estimate the full covariance of the resulting two-mode state by observing its transient decay during subsequent free evolution.