Considering gene flow when using coalescent methods to delimit lineages of North American pitvipers of the genus Agkistrodon

Examining species diversity and mechanisms of speciation using coalescent models provides a framework for how regional diversity is accrued, even in well-studied areas such as the Nearctic. It is likely, that gene flow among closely-related species with adjacent distributions may be common. However, the absence of gene flow is a primary assumption of many phylogeographical methods that produce species trees and delimit species using Bayesian or likelihood functions in a coalescent framework. In the present study, we examine delimitation when gene flow between species is present using empirical datasets from two species of North American pitvipers of the genus Agkistrodon. We also use niche modelling to determine whether these young lineages occur in distinct environmental niches. To manage the problem of gene flow between species, we first identify admixed individuals, demonstrate that gene flow has occurred, and then identify the impact of alternative population assignments of admixed individuals on delimitation posterior probabilities. In addition, we examine the influence of mitochondrial genes relative to other loci combined in coalescent analyses that delimit species. Here, we find that the copperheads (Agkistrodon contortrix) and the cottonmouths (Agkistrodon piscivorus) are each composed of two distinct species, with each occupying different niches. Importantly, we find that species can be delimited when the amount of gene flow between lineages is low, although the methods are acutely sensitive to population assignment of individuals.(c) 2014 The Linnean Society of London