4.5 Article

Examiner characteristics and interrater reliability in a communication OSCE

Journal

PATIENT EDUCATION AND COUNSELING
Volume 100, Issue 6, Pages 1230-1234

Publisher

ELSEVIER IRELAND LTD
DOI: 10.1016/j.pec.2017.01.013

Keywords

Communication competence; Assessment; Examiner factors; OSCE

Ask authors/readers for more resources

Objective: To identify inter-individual examiner factors associated with interrater reliability in a summative communication OSCE in the 4th study year. Methods: The OSCE consists of 4 stations assessed with a 4-item 5-point global rating instrument. A bivariate secondary analysis of interrater reliability in relation to 4 examiner factors (gender, profession, OSCE experience, examiner training) was conducted. Intraclass correlation coefficients (ICC) were calculated and compared between examiner dyads of different similarity. Results: 169 pairwise ratings from 19 different examiners in 16 dyads were analysed. Interrater reliability is significantly higher in examiner dyads of same vs. different gender (ICC = 0.76 (95% CI = 0.65-0.83) vs. ICC = 0.41 (95% CI = 0.21-0.57)), in dyads of two clinicians vs. non-clinical/mixed professions (ICC = 0.72 (95% CI = 0.56-0.83) vs. ICC = 0.57 (95% CI = 0.41-0.69)), and in dyads with high vs. low/mixed OSCE experience (ICC = 0.73 (95% CI 0.50-0.87) vs. ICC = 0.56 (95% CI = 0.41-0.69)). Participation in recent examiner training had no influence on ICCs. Conclusion: Better concordance of ratings between clinically active examiners might be a hint for context specificity of good communication. Higher interrater reliability between examiners with same gender may indicate gender-specific communication concepts. Practice implications: Medical faculties introducing summative assessment of communication competence should focus the influence of examiner characteristics on interrater reliability. (C) 2017 Elsevier B.V. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available