Incommensurate magnetic order and phase separation in the two-dimensional Hubbard model with nearest- and next-nearest-neighbor hopping

We consider the ground-state magnetic phase diagram of the two-dimensional Hubbard model with nearest- and next-nearest-neighbor hopping in terms of electronic density and interaction strength. We treat commensurate ferromagnetic and antiferromagnetic as well as incommensurate (spiral) magnetic phases. The first-order magnetic transitions with changing chemical potential, resulting in a phase separation (PS) in terms of density, are found between ferromagnetic, antiferromagnetic, and spiral magnetic phases. We argue that PS has a dramatic influence on the phase diagram in the vicinity of half-filling. The results imply possible interpretation of the unusual behavior of magnetic properties of single-layer cuprates in terms of PS between collinear and spiral magnetic phases. The relevance of our results to the magnetic properties of the ruthenates is also discussed.