Use of an Automated Case Log to Improve Trainee Evaluations on a Pediatric Emergency Medicine Rotation

Objective: Providing meaningful evaluation to trainees rotating through the pediatric emergency medicine is important yet challenging. Information systems can be used to autopopulate an electronic case log, which can be leveraged to assist in the evaluation process. The objective of this study was to determine if a novel educational initiative using an automated case log improved faculty evaluation of trainees. Methods: This retrospective study examined faculty completion rate, as well as the content of medical student evaluations over a 3-academic-year study period. Three phases of evaluation were utilized: written, electronic, and electronic enhanced with individualized case reports created with the automated case log. The primary outcome was faculty response rate. Secondary outcomes included word count and the number of themes identified following qualitative analysis of narrative responses. Logistic regression was performed. Results: Forty-one faculty members completed evaluations of 43 students. The rates of completion for the written, electronic, and automated case log phases were 18%, 16%, and 62%, respectively. Faculty in the automated case log phase were significantly more likely to complete evaluations compared with those in the written evaluation phase (odds ratio, 7.6; 95% confidence interval, 4.5-13.0). The median word counts across the 3 phases were 19, 36, and 43, respectively. The median numbers of themes identified during the 3 phases were 3, 4, and 5, respectively. The differences in the word count and median number of themes between the written and automated case log phases were significantly different (P < 0.001). Conclusions: The process of trainee evaluation can be improved by utilizing an automated case log to provide faculty members with individualized reports of shared patient encounters.