Instability of a hybrid module of antagonistic and mutualistic interactions

Mutualistic and antagonistic interactions coexist in nature. However, little is understood about their relative roles and interactive effects on multispecies coexistence. Here, using a three-species population dynamics model of a resource species, its exploiter, and a mutualist species, we show that a mixture of different interaction types may lead to dynamics that differ completely from those of the isolated interacting pairs. More specifically, a combination of globally stable antagonistic and mutualistic subsystems can lead to unstable population oscillations, suggesting the potential difficulty in the coexistence of antagonism and mutualism. Mutualism-induced instability arises from the indirect positive effect of mutualism on the exploiter. Furthermore, for a three-species system with a stronger mutualistic interaction to persist stably, a weaker antagonistic interaction is required. Network studies of communities composed of one type of interaction may not capture the dynamics of natural communities.