Autocorrelation and estimates of treatment effect size for single-case experimental design data

We examined the degree of autocorrelation among single-case design data with six measures used to estimate treatment effect size. The most commonly used measures of effect size for single-case data over the last 5 years published in peer-reviewed journals were selected for comparison. The overall mean degree of autocorrelation was 0.46 (SD = 0.33) across the 304 data paths, which represents a moderate degree of autocorrelation. Overall, it appears that non-parametric measures of effect size (i.e., percent of non-overlapping data [PND], non-overlap of all pairs [NAP], and improvement rate difference [IRD] values) were substantially and significantly more influenced by the degree of autocorrelation. Tau-U effect size estimate was the non-parametric exception as it was not significantly influenced by the degree of autocorrelation. Parametric measures of effect sizes such as standardized mean difference (SMD) and log response ratio (LRR) values did not appear to be significantly influenced by the degree of autocorrelation. For SMD, LRR, and Tau-U values, the correlation between the effect size value and the degree of autocorrelation was minimal. For NAP, IRD, and PND values, the correlation between the effect size value and the degree of autocorrelation was moderate, indicating that these estimates of effect size should be avoided as the degree of autocorrelation between data points increases.

Automated summary

The study found that non-parametric effect size measures are more significantly influenced by the degree of autocorrelation compared to parametric measures. It is recommended to avoid using NAP, IRD, and PND values as the autocorrelation between data points increases in single-case design data analysis.