Extinction dynamics from metastable coexistences in an evolutionary game

Deterministic evolutionary game dynamics can lead to stable coexistences of different types. Stochasticity, however, drives the loss of such coexistences. This extinction is usually accompanied by population size fluctuations. We investigate the most probable extinction trajectory under such fluctuations by mapping a stochastic evolutionary model to a problem of classical mechanics using the Wentzel-Kramers-Brillouin (WKB) approximation. Our results show that more abundant types in a coexistence may be more likely to go extinct first, in good agreement with previous results. The distance between the coexistence and extinction points is not a good predictor of extinction either. Instead, the WKB method correctly predicts the type going extinct first.