Journal
BEHAVIOR RESEARCH METHODS
Volume -, Issue -, Pages -Publisher
SPRINGER
DOI: 10.3758/s13428-022-02043-8
Keywords
Missing data; Average treatment effect; Standardized mean difference; Maximum likelihood; Multiple imputation
Categories
Ask authors/readers for more resources
Reporting standardized effects is important in randomized treatment studies. However, estimating the standardized average treatment effect (sATE) accurately becomes challenging when outcome data are missing. In this study, different strategies for handling missing data were compared using a simulation study, and recommendations for improving estimation were provided.
Reporting standardized effects in randomized treatment studies aids interpretation and facilitates future meta-analyses and policy considerations. However, when outcome data are missing, achieving an unbiased, accurate estimate of the standardized average treatment effect, sATE, can pose challenges even for those with general knowledge of missing data handling, given that the sATE is a ratio of a mean difference to a (within-group) standard deviation. Under both homogeneity and heterogeneity of variance, a Monte Carlo simulation study was conducted to compare missing data handling strategies in terms of bias and accuracy in the sATE, under specific missingness patterns plausible for randomized pretest posttest studies. Within two broad missing data handling approaches, maximum likelihood and multiple imputation, modeling choices were thoroughly investigated including the analysis model, variance estimator, imputation algorithm, and method of pooling results across imputed datasets. Results demonstrated that although the sATE can be estimated with little bias using either maximum likelihood or multiple imputation, particular attention should be paid to the model and variance estimator, especially at smaller sample sizes (i.e., N = 50). Differences in accuracy were driven by differences in bias. To improve estimation of the sATE in practice, recommendations and a software demonstration are provided. Moreover, a pedagogical explanation of the causes of bias, described separately for the numerator and denominator of the sATE is provided, demonstrating visually how and why bias occurs with certain methods.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available