期刊
JOURNAL OF SURGICAL EDUCATION
卷 79, 期 6, 页码 E263-E272出版社
ELSEVIER SCIENCE INC
DOI: 10.1016/j.jsurg.2020.09.022
关键词
Video commentary; assessment; machine learning; artificial intelligence; surgical curriculum; medical education
资金
- Association of Program Directors in Surgery (APDS) Research Award
Integrating machine learning into medical education, especially in teaching and assessment of complex problem solving, has great potential. This study applied machine learning to predict the training level of surgical residents using a structured video commentary assessment, achieving better predictive performance compared to traditional statistical analysis.
BACKGROUND & OBJECTIVE: Teaching and assessment of complex problem solving are a challenge for medical education. Integrating Machine Learning (ML) into medical education has the potential to revolutionize teaching and assessment of these problem-solving processes. In order to demonstrate possible applications of ML to education, we sought to apply ML in the context of a structured Video Commentary (VC) assessment, using ML to predict residents' training level. SETTING: A secondary analysis of multi-institutional, IRB approved study. Participants had completed the VC assessment consisting of 13 short (20-40 seconds) operative video clips. They were scored in real-time using an extensive checklist by an experienced proctor in the assessment. A ML model was developed using Tensor-Flow and Keras. The individual scores of the 13 video clips from the VC assessment were used as the inputs for the ML model as well as for regression analysis. PARTICIPANTS: A total of 81 surgical residents of all postgraduate years (PGY) 1-5 from 7 institutions constituted the study sample. RESULTS: Scores from individual VC clips were strongly positively correlated with PGY level (p = 0.001). Some video clips were identified to be strongly correlated with a higher total score on the assessment; others had significant influence when used to predict trainees' PGY levels. Using a supervised machine learning model to predict trainees' PGY resulted in a 40% improvement over traditional statistical analysis. CONCLUSIONS: Performing better in a few select video clips was key to obtaining a higher total score but not necessarily foretelling of a higher PGY level. The use of the total score as a sole measure may fail to detect deeper relationships. Our ML model is a promising tool in gauging learners' levels on an assessment as extensive as VC. The model managed to approximate residents' PGY levels with a lower MAE than using traditional statistics. Further investigations with larger datasets are needed. (C) 2020 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据