Efficacy and Feasibility of Objective Structured Clinical Examination in the Internal Assessment for Surgery Postgraduates

INTRODUCTION: Traditionally assessment in medical training programs has been through subjective faculty evaluations or multiple choice questions. Conventional examinations provide assessment of the global performance rather than individual competencies thus making the final feedback less meaningful. The objective structured clinical examination (OSCE) is a relatively new multidimensional tool for evaluating training. This study was carried out to determine the efficacy and feasibility of OSCE as a tool for the internal assessment of surgery residents. METHODS: This study was carried out on the marks obtained by surgery residents at different levels of training in a single tertiary center in India over the 4 OSCEs conducted in the years 2015 and 2016. The marks of the OSCE were collected from the departmental records and analyzed. Reliability was calculated using internal consistency using Cronbach's alpha. Validity was calculated by item total correlation. Content validation was done by obtaining expert reviews from 5 experts using a proforma, to assess the content and checklist of each station of the OSCE. RESULTS: A total of 49 surgery residents were assessed in small batches during the above mentioned period. Of the 4 OSCEs conducted by us, 3 had a high value of Cronbach's alpha of greater than 0.9, as opposed to the set standard of 0.7. Out of 23 stations used in the 4 examinations separately, only 3 stations were found to have a low correlation coefficient (item total correlation), and hence, a low validity. The remaining 20 stations were found to have a high validity. Expert review showed unanimous validation of the content of 17 out of the 23 stations, with few suggestions for change in the remaining 6 stations. The material and manpower used was minimal and easy to obtain, thus making the OSCE feasible to conduct. CONCLUSION: OSCE is a reliable, valid. and feasible method for evaluating surgery residents at various levels of training. ((C) 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)