期刊
DIAGNOSTICA
卷 50, 期 2, 页码 65-77出版社
HOGREFE & HUBER PUBLISHERS
DOI: 10.1026/0012-1924.50.2.65
关键词
performance assessment; rater agreement; rater severity; Rasch model; language performance
Performance assessments are subject to rater biases which may operate to strongly reduce the assessments' precision and validity. One particularly influential rater bias refers to the severity, or leniency, of ratings. The present research employs a many-faceted Rasch model (Linacre, 1989; Linacre & Wright, 2002) which allows to measure each rater's severity and to put these severity measures into a common frame of reference along with measures of examinee proficiency and task or criteria difficulty. Moreover, this model yields ability estimates corrected for differences in rater severity. In an application of this approach to the Test of German as a Foreign Language (Test Deutsch als Fremdsprache, TestDaF), ratings of the writing performance of 1359 examinees given by 2 raters each out of a total of 29 raters on 3 criteria are closely examined. The group of raters is shown to be highly heterogeneous, necessitating a correction of assessments for rater severity bias. Finally, various implications of the many-faceted Rasch measurement approach to the evaluation of performance assessments are discussed.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据