Standardized or real patients to test clinical competence? The long case revisited

Background In undergraduate clinical examinations, the use of real patients as long cases is being replaced by objective structured clinical examinations (OSCEs) which use simulated scenarios, although we lack published psychometric data on long cases to support the move from real to simulated patients. Aim To assess candidate performance across two history-taking long cases to estimate the number of cases required for a reliable assessment. Results are compared with psychometric data from an OSCE. Setting A final-year qualifying undergraduate clinical examination. Method Two observed history-taking long cases were included, alongside an OSCE. Candidates interviewed two unstandardized real patients. The history-taking part (14 minutes) was observed, uninterrupted, by examiner(s) who assessed data gathering, interviewing and diagnostic and management skills. The presentation (7 minutes) was unstructured; the examiner(s) intervened as appropriate. Marks were expressed as a percentage of the total possible score and analysed using generalizability theory to estimate intercase reliability. Results Two examiner pairs independently rated both long cases for 79 (36.7%) of the 214 candidates. Projections based on generalizability theory showed that 10 20-minute cases would give reliabilities of 0.84 for single-marked and 0.88 for double-marked candidates, compared with a projected reliability of 0.73 for the same 214 candidates taking the OSCE. Conclusion If history-taking long cases are observed, three-and-a-half hours of testing time using 10 unstandardized patients would produce a reliable test. Long cases therefore are, in terms of reliability, no worse and no better than OSCEs in assessing clinical competence.