期刊
LANGUAGE TESTING
卷 28, 期 4, 页码 543-560出版社
SAGE PUBLICATIONS LTD
DOI: 10.1177/0265532211406422
关键词
inter-/intra-rater reliability; multi-faceted Rasch analysis; rater development; rater expertise; rater training; writing assessment
Raters are central to writing performance assessment, and rater development - training, experience, and expertise - involves a temporal dimension. However, few studies have examined new and experienced raters' rating performance longitudinally over multiple time points. This study uses operational data from the writing section of the MELAB (n = 20,662 ratings), an international exam of English proficiency, to investigate the rating quality of new and experienced raters over three time periods of 12 to 21 months. Rating quality was operationalized in terms of rater severity and consistency, and estimates of those modeled using multi-facet Rasch methodology. Results indicate that, within one particular rating context, (1) novice raters, where initially differing in performance, learn to rate appropriately relatively quickly, (2) raters are able to maintain rating quality over time, and (3) rating volume and rating quality may be related. Implications for rater preparation, rater certification, and the notion of expert rater are discussed.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据