Matching Methods for Clustered Observational Studies in Education

Many interventions in education occur in settings where treatments are applied to groups. For example, a reading intervention may be implemented for all students in some schools and withheld from students in other schools. When such treatments are nonrandomly allocated, outcomes across the treated and control groups may differ due to the treatment or due to baseline differences between groups. When this is the case, researchers can use statistical adjustment to make treated and control groups similar in terms of observed characteristics. Recent work in statistics has developed matching methods designed for contexts where treatments are clustered. This form of matching, known as multilevel matching, may be well suited to many education applications where treatments are assigned to schools. In this article, we provide an extensive evaluation of multilevel matching and compare it to multilevel regression modeling. We evaluate multilevel matching methods in two ways. First, we use these matching methods in a within-study comparison design and attempt to recover treatment effect estimates from three clustered randomized trials. Second, we conduct a simulation study. We find evidence that generally favors an analytic approach to statistical adjustment that combines multilevel matching with regression adjustment. We conclude with an empirical application.

Automated summary

This article explores the use of multilevel matching methods in educational settings and compares it to multilevel regression modeling. The study finds evidence supporting an analytic approach that combines multilevel matching with regression adjustment for treatment effect estimation. The results suggest that multilevel matching may be effective in making treated and control groups similar in terms of observed characteristics.