Journal
BEHAVIOR RESEARCH METHODS
Volume 45, Issue 3, Pages 792-812Publisher
SPRINGER
DOI: 10.3758/s13428-012-0281-2
Keywords
Analysis of variance; Robustness; Nonnormality; Small sample settings; Repeated measurements; Correlated data; Multivariate; Mixed model analyses; Multilevel model; Simulation study; Type I error rate; Central limit theorem; Monte Carlo
Categories
Ask authors/readers for more resources
Repeated measures analyses of variance are the method of choice in many studies from experimental psychology and the neurosciences. Data from these fields are often characterized by small sample sizes, high numbers of factor levels of the within-subjects factor(s), and nonnormally distributed response variables such as response times. For a design with a singlewithin-subjects factor, we investigated Type I error control in univariate tests with corrected degrees of freedom, the multivariate approach, and a mixed-model (multilevel) approach (SAS PROC MIXED) with Kenward-Roger's adjusted degrees of freedom. We simulated multivariate normal and nonnormal distributions with varied population variance-covariance structures (spherical and nonspherical), sample sizes (N), and numbers of factor levels (K). For normally distributed data, as expected, the univariate approach with Huynh-Feldt correction controlled the Type I error rate with only very few exceptions, even if samples sizes as low as threewere combined with high numbers of factor levels. The multivariate approach also controlled the Type I error rate, but it requires N >= K. PROC MIXED often showed acceptable control of the Type I error rate for normal data, but it also produced several liberal or conservative results. For nonnormal data, all of the procedures showed clear deviations from the nominal Type I error rate in many conditions, even for sample sizes greater than 50. Thus, none of these approaches can be considered robust if the response variable is nonnormally distributed. The results indicate that both the variance heterogeneity and covariance heterogeneity of the population covariance matrices affect the error rates.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available