Diagnostic testing and educational assessment

Background: Clinicians are familiar with making diagnostic decisions based on information gathered from history, clinical examination and diagnostic tests. Although many clinicians assess students, they may be less familiar with ways to assimilate assessment information to inform educational decisions. We draw parallels between the processes used to make a clinical diagnosis and the similar processes needed to make an educational decision. Context: There are several indices that describe the performance and utility of diagnostic tests, which we have extrapolated to educational assessment. Innovation: We provide a clinical diagnostic question and an education assessment question, and use examples of indices of performance and utility for both of these situations to explore: reliability, indeterminate results, certainty in decisions, acceptable levels of sensitivity and specificity, pre-test probability and dealing with limitations. Test reliability requires adequate sampling and consistency between observers. Seeking more information should be targeted to situations where decisions are not certain. Altering score cutpoints alters test sensitivity and specificity, which in assessment will alter the numbers of falsely passing or falsely failing candidates. Just as the pre-test probability of a diagnosis influences how to interpret diagnostic tests, so too does the pretest probability of failure alter the performance characteristics of assessments. In clinical situations, a 'wait and see' approach may be limited by clinical urgency. Likewise, in assessment the 'wait and see' approach may be limited by a duty to society. Implications: Clinicians familiar with the performance and utility of diagnostic tests can extrapolate that knowledge to make better interpretations of educational assessments.