Conjunctive standards in OSCEs: The why and the how of number of stations passed criteria

Introduction Many institutions require candidates to achieve a minimum number of OSCE stations passed (MNSP) in addition to the aggregate pass mark. The stated rationale is usually that this conjunctive standard prevents excessive degrees of compensation across an assessment. However, there is a lack of consideration and discussion of this practice in the medical education literature. Methods We consider the motivations for the adoption of the MNSP from the assessment designer perspective, outlining potential concerns about the complexity of what the OSCE is trying to achieve, particularly around the blueprinting process and the limitations of scoring instruments. We also introduce four potential methods for setting an examinee-centred MNSP standard, and highlight briefly the theoretical advantages and disadvantages of these approaches. Discussion and conclusion There are psychometric arguments for and against the limiting of compensation in OSCEs, but it is clear that many stakeholders value the application of an MNSP standard. This paper adds to the limited literature on this important topic and notes that current MNSP practices are often problematic in high stakes settings. More empirical work is needed to develop understanding of the impact on pass/fail decision-making of the proposed standard setting methods developed in this paper.

Automated summary

This paper explores the motivations behind the adoption of the MNSP in OSCEs and introduces potential methods for setting an examinee-centred standard. It discusses the psychometric arguments for and against limiting compensation in OSCEs, emphasizing the value many stakeholders place on an MNSP standard. The paper also highlights the need for more empirical work to understand the impact of different standard setting methods on pass/fail decision-making in high stakes settings.