Comparing parent-offspring regression with frequentist and Bayesian animal models to estimate heritability in wild populations: a simulation study for Gaussian and binary traits

Estimating heritability of traits in wild populations is a major prerequisite to understand their evolution. Until recently, most heritability estimates had been obtained using parent-offspring regressions. However, the popularity of animal models, that is, (generalized) linear mixed models assessing the genetic variance component based on population pedigree information, has markedly increased in the past few years. Animal models are claimed to perform better than parentoffspring regressions mainly because they use full between-individual relatedness information and they allow explicit modelling of the environmental effects shared by individuals. However, the differences between heritability estimates obtained using both approaches are not straight forward, and the factors influencing these differences remain unclear. We performed a simulation study to evaluate and compare the accuracy and precision of estimates obtained from parentoffspring regressions and animal models using both Frequentist (REML, PQL) and Bayesian (MCMC) estimation methods. We explored the influence of (i) the presence and type of shared environmental effects (non-transgenerational or transgenerational), (ii) the distribution of the phenotypic trait considered (Gaussian or binary trait) and (iii) data quantity and quality (sample size, pedigree connectivity) on heritability estimates obtained from the two approaches for different levels of true heritability. In the absence of shared environmental effects, the animal model using the REML method performed best for a Gaussian trait, while the animal model using MCMC was more appropriate for a binary trait. For low quantity and quality data, and a binary trait, the parentoffspring regression yielded very imprecise estimates. Estimates from the parentoffspring regression were not influenced by a non-transgenerational shared environmental effect, whereas estimates from animal models in which environmental effects are ignored were affected by both non-transgenerational and transgenerational effects. We discuss the relevance of each approach and estimation method for estimating heritability in wild populations. Importantly, because most effects fitted in animal models are, in fact, non-transgenerational (including environmental maternal effects), we advocate a systematic comparison between parentoffspring regression and animal model estimates to detect potentially missing non-transgenerational environmental effects.