Evaluating the management of septic shock using patient simulation

Objective: Develop a scoring system that can assess the management of septic shock by individuals and teams. Design: Retrospective review of videotapes of critical care house staff managing a standardized simulation of septic shock. Setting: Academic medical center; videotapes were made in a recreated intensive care unit environment using a high-fidelity patient simulator. Subjects: Residents in medicine, surgery, and anesthesiology who had participated in the intensive care unit rotation. Interventions: The septic patient was managed by the intensive care unit team in a graded manner with interns present for the first 10 mins and more senior-level help arriving after 10 mins. The intern was graded separately for the first 10 mins, and the team was graded for the entire 35-min performance. Measurements and Main Results: Both technical and nontechnical scoring systems were developed to rate the management of septic shock. Technical scores are based on guidelines and principles of managing septic shock. Team leadership, communication, contingency planning, and resource utilization were addressed by the nontechnical rating. Technical scores were calculated for both interns and teams; nontechnical scores applied only to the team. Of 16 technical checklist items, interns completed a mean of 7 with a range of 1.5-11. Team technical ratings had a mean of 9.3 with a range of 3.3-13. Nontechnical scores showed similar intergroup variability with a mean of 26 and a range of 10-35. Technical and nontechnical scores showed a modest correlation (r = .40, p = .05). Interrater reliabilities for intern and team technical scores were both r = .96 and for nontechnical scores r = .88. Conclusions: Objective measures of both knowledge-based and behavioral skills pertinent to the management of septic shock were made. Scores identified both adequate and poor levels of performance. Such assessments can be used to benchmark clinical skills of individuals and groups over time and may allow the identification of interventions that improve clinical effectiveness in sepsis management.