Reporting Proficiency Levels for Examinees With Incomplete Data

Takers of educational tests often receive proficiency levels instead of or in addition to scaled scores. For example, proficiency levels are reported for the Advanced Placement (AP(R)) and U.S. Medical Licensing examinations. Technical difficulties and other unforeseen events occasionally lead to missing item scores and hence to incomplete data on these tests. The reporting of proficiency levels to the examinees with incomplete data requires estimation of the performance of the examinees on the missing part and essentially involves imputation of missing data. In this article, six approaches from the literature on missing data analysis are brought to bear on the problem of reporting of proficiency levels to the examinees with incomplete data. Data from several large-scale educational tests are used to compare the performances of the six approaches to the approach that is operationally used for reporting proficiency levels for these tests. A multiple imputation approach based on chained equations is shown to lead to the most accurate reporting of proficiency levels for data that were missing at random or completely at random, while the model-based approach of Holman and Glas performed the best for data that are missing not at random. Several recommendations are made on the reporting of proficiency levels to the examinees with incomplete data.

Automated summary

This article explores the reporting of proficiency levels to examinees with incomplete data through the comparison of seven different approaches. The multiple imputation approach based on chained equations is found to be the most accurate for data missing at random or completely at random, while the model-based approach by Holman and Glas performs best for data missing not at random. Several recommendations are made for reporting proficiency levels to examinees with incomplete data.