Adequacy of approximations to distributions of test statistics in complex mixed linear models

A recent study of lady beetle antennae was a small sample repeated measures design involving a complex covariance structure. Distributions of test statistics based on mixed models fitted to such data are unknown, but two recently developed methods for approximating the distributions of test statistics in mixed linear models have been included as options in the latest release of the MIXED procedure of SAS(R). One method (FC, from Fai and Cornelius) computes degrees of freedom of an approximating F distribution for the test statistic using spectral decomposition of the hypothesis matrix together with repeated application of a method for single-degree-of-freedom tests. The other method (KR, from Kenward and Roger) adjusts the estimated covariance matrix of the parameter estimates, computes a scale adjustment to the test statistic, and computes the degrees of freedom of an approximating F distribution. Using the two methods, p values for a hypothesis of interest in the lady beetle study Were quite different. Simulation studies on the Proc MIXED implementation of these methods showed that Type I error rates of both methods are affected by covariance structure complexity, sample size, and imbalance. Nonetheless, the KR method performs well in situations with fairly complicated covariance structures when sample sizes are moderate to small and the design is reasonably balanced. The KR method should be used in preference to the FC method, although it had inflated Type I error rates for complex covariance structures combined with small sample sizes.