Permutation tests for phylogenetic comparative analyses of high-dimensional shape data: What you shuffle matters

Evaluating statistical trends in high-dimensional phenotypes poses challenges for comparative biologists, because the high-dimensionality of the trait data relative to the number of species can prohibit parametric tests from being computed. Recently, two comparative methods were proposed to circumvent this difficulty. One obtains phylogenetic independent contrasts for all variables, and statistically evaluates the linear model by permuting the phylogenetically independent contrasts (PICs) of the response data. The other uses a distance-based approach to obtain coefficients for generalized least squares models (D-PGLS), and subsequently permutes the original data to evaluate the model effects. Here, we show that permuting PICs is not equivalent to permuting the data prior to the analyses as in D-PGLS. We further explain why PICs are not the correct exchangeable units under the null hypothesis, and demonstrate that this misspecification of permutable units leads to inflated type I error rates of statistical tests. We then show that simply shuffling the original data and recalculating the independent contrasts with each iteration yields significance levels that correspond to those found using D-PGLS. Thus, while summary statistics from methods based on PICs and PGLS are the same, permuting PICs can lead to strikingly different inferential outcomes with respect to statistical and biological inferences.