Measurements of performance gaps are sensitive to the level of test stakes: Evidence from PISA and a Field Experiment

Educational performance gaps are a long-lasting issue of concern in many countries. Many times, these gaps are measured using low-stakes tests that are especially sensitive to motivation and test engagement. I conduct a field experiment in 7 Israeli Jewish and Arab middle schools and use data from PISA 2015 to investigate whether part of the performance gaps between ethnic minority and majority groups could be attributed to test engagement rather than proficiency. I find that test engagement can account for more than 50% of the performance gaps measured in low-stakes assessments. Test engagement is also important, as it correlates with several personality traits important to life outcomes. Therefore, considering the importance of both test engagement, which influences low-stakes assessments, and proficiency, which is more accurately discerned in high-stakes assessments, policymakers should make deliberate decisions regarding the stakes assigned to assessment tests, taking into account the specific evaluation criteria that they aim to emphasize.

Automated summary

This study conducted a field experiment in Israeli middle schools and used data from PISA 2015 to examine the impact of test engagement on performance gaps between ethnic minority and majority groups. The findings suggest that test engagement can account for more than 50% of the performance gaps measured in low-stakes assessments. Test engagement is also correlated with personality traits that influence life outcomes. Policymakers should consider both test engagement and proficiency when assigning stakes to assessment tests.