Computation and Accuracy Evaluation of Comparable Scores on Culturally Responsive Assessments

Culturally responsive assessments have been proposed as potential tools to ensure equity and fairness for examinees from all backgrounds including those from traditionally underserved or minoritized groups. However, these assessments are relatively new and, with few exceptions, are yet to be implemented in large scale. Consequently, there is a lack of guidance on how data on how one can compute comparable scores on various versions of these assessments. In this paper, the multigroup multidimensional Rasch model is repurposed for modeling data originating from various versions of a culturally responsive assessment and for analyzing such data to compute comparable scores. Two simulation studies are performed to evaluate the performance of the model for data simulated from hypothetical culturally responsive assessments and to find the conditions under which the computed scores are accurate. Recommendations are made for measurement practitioners interested in culturally responsive assessments.

Automated summary

This paper introduces a method of reusing the multigroup multidimensional Rasch model to model and analyze data from culturally responsive assessments, and provides conditions for computing comparable scores. The performance of the model on simulated data from hypothetical culturally responsive assessments is evaluated, and recommendations are made for measurement practitioners interested in this area.