The role of student effort on performance in PISA: revisiting the gender gap in achievement

International assessments are important to benchmark the quality of education internationally. However, ignoring students' efforts in low-stakes tests can lead to biased interpretations of test performance. We use data from the PISA to analyse the potential role of student effort in explaining gender achievement gaps across countries. We build two effort measures based on students' response times to test questions (i.e. rapid-guessing rates) and the student effort on the post-test survey (i.e. item non-response rates). We find a strong association between student efforts and test scores. After accounting for differences in effort, the estimated gender achievement gap in mathematics and science could be up to 36% and 40% of an standard deviation wider, respectively, in favour of boys. Altogether, our effort measures on average explain between 51% and 56% of the cross-country variation in test scores.

Automated summary

International assessments are crucial for comparing education quality worldwide, yet disregarding students' efforts in low-stakes tests can result in biased interpretations of test performance. By utilizing data from the PISA, we investigate the potential role of student effort in explaining gender achievement gaps across countries. Two effort measures, based on students' response times and post-test survey, highlight a strong association between student efforts and test scores. Accounting for effort differences, the estimated gender achievement gaps in mathematics and science could be significantly wider, favoring boys by up to 36% and 40% of a standard deviation, respectively. Overall, our effort measures explain an average of 51% to 56% of the cross-country variation in test scores.