Spin-orbital frustration in molybdenum pyrochlores A2Mo2O7 (A = rare earth)

Electronic and magnetic properties of molybdenum pyrochlores A(2)Mo(2)O(7) are studied by the fully relativistic density-functional theory plus on-site repulsion (U) method, with a focus on the spin-glass insulating material Y2Mo2O7. We find that the system exhibits peculiar competition in energy between different magnetic states in the large-U insulating region. The magnetic competition cannot be explained by the conventional picture based on the geometrical frustration of isotropic Heisenberg antiferromagnetic exchange interactions. Through an analysis by using a generalized spin model, we find that the effective spin interactions are distinct from the simple Heisenberg form and strongly anisotropic in spin space. We also reveal that they give rise to keen competition between antiferromagnetic and ferromagnetic states. The complex form of the magnetic interactions indicates a crucial role of the orbital degree of freedom. Analyzing a three-orbital Hubbard model, we clarify that the magnetic competition is tightly connected with orbital frustration in the 4d(2) electronic configuration through the spin-orbital interplay. The results challenge the conventional picture of the spin-glass behavior that attributes the origin to the geometrical frustration of purely antiferromagnetic exchange interactions.