Fixation probabilities in populations under demographic fluctuations

We study the fixation probability of a mutant type when introduced into a resident population. We implement a stochastic competitive Lotka-Volterra model with two types and intra-and interspecific competition. The model further allows for stochastically varying population sizes. The competition coefficients are interpreted in terms of inverse payoffs emerging from an evolutionary game. Since our study focuses on the impact of the competition values, we assume the same net growth rate for both types. In this general framework, we derive a formula for the fixation probability. of the mutant type under weak selection. We find that the most important parameter deciding over the invasion success of the mutant is its death rate due to competition with the resident. Furthermore, we compare our approximation to results obtained by implementing population size changes deterministically in order to explore the parameter regime of validity of our method. Finally, we put our formula in the context of classical evolutionary game theory and observe similarities and differences to the results obtained in that constant population size setting.