Selection dynamics and limits under additive x additive epistatic gene action

Theory predicts that response to selection under additive gene action with finite population size will decline over cycles of selection. In practice, this decline is not always observed. Selection under additive X additive epistatic gene action was examined to determine if it could condition greater and longer-term response to selection than additive gene action relative to the initial response in the base population. By use of transition matrix simulation, the ratio of the long-term to the initial selection response under epistatic gene action was found to be more than double that under additive gene action for effective population sizes of N = 2, 4, and 8. Furthermore, the selection half-lives under epistatic and additive models approximated 2N and 1.4N, respectively, indicating that selection response lasts longer under epistatic than additive gene action. Analysis showed that under the epistatic model, selection affected the mean in two distinct ways. First, selection generated a positive covariance in the allele content among selected individuals. This effect was strengthened by linkage between interacting loci. Second, selection shifted allele frequencies to increase covariance in allele frequencies among selection lines. This effect was weakened by linkage between interacting loci. At intermediate allele frequencies, initial selection response was due entirely to the first effect and was stronger for small than for large N. Consequently, by the third cycle of selection, response was 76% greater for N = 2 than for N = 8 under an infinitesimal epistatic model with unlinked loci and selection intensity of 50%. This differentiation in response between population sizes may lead to an experimental test of the importance of epistatic gene action affecting a trait.