期刊
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT
卷 78, 期 3, 页码 430-459出版社
SAGE PUBLICATIONS INC
DOI: 10.1177/0013164416689696
关键词
rater effects; social network analysis; exponential random graph models; Rasch model
It is common practice for assessment programs to organize qualifying sessions during which the raters (often known as markers or judges) demonstrate their consistency before operational rating commences. Because of the high-stakes nature of many rating activities, the research community tends to continuously explore new methods to analyze rating data. We used simulated and empirical data from two high-stakes language assessments, to propose a new approach, based on social network analysis and exponential graph models, to evaluate the readiness of a group of raters for operational rating. The results of this innovative approach are compared with the results of a Rasch analysis, which is a well-established approach for the analysis of such data. We also demonstrate how the new approach can be practically used to investigate important research questions such as whether rater severity is stable across rating tasks. The merits of the new approach, and the consequences for practice are discussed.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据