Construct validation of a 3D printed neonatal thoracoscopic simulator: Can it measure expertise?

Background: acquiring technical expertise for neonatal thoracoscopy is challenging. To address this, we designed a fully synthetic thoracoscopic simulator for which we established its construct validity. Methods: three thoracoscopic tasks were assessed: ring transfer, needle pass and incision of a blind upper esophageal pouch (EA cut). Participants watched instructional videos with accompanying written instruc-tions for each task before having their attempt video recorded. All tasks were marked by three blinded pediatric surgeons using a modified Objective Structured Assessment of Technical Skills (OSATS). Scores were assessed using appropriate statistical analysis and inter-rater reliability was analyzed by interclass correlation coefficient (ICC). Results: 23 participants completed the ring transfer and needle pass and 21 the EA cut: 5 experts (con -sultant surgeons), 5 intermediate (registrars on a training program) and 13 novices (medical students, house surgeons or non-training registrars). All three tasks distinguished between novice and interme-diate/expert (ring transfer p = 0.0 0 0 01, needle pass p = 0.0 0 04 and EA cut p = 0.0014, respectively). Interrater reliability was good for ring transfer and needle pass but poor for EA cut. Conclusion: the tasks distinguished between novice and intermediate/expert but not between expert and intermediate. In needle pass and EA cut, there was a trend for the experts to score higher than inter-mediate participants. Ring transfer and needle pass tasks achieved construct validity, had good interrater reliability and were found to be useful in assessing a novice surgeon's progression towards the interme-diate level. Distinguishing between intermediate and expert may require assessment of more complex tasks such as intracorporeal suturing and tying. Level of evidence: II (c) 2021 Elsevier Inc. All rights reserved.

Automated summary

Through the assessment of three thoracoscopic tasks, we found that these tasks could effectively differentiate between novices and intermediate/expert levels, but not between experts and intermediate participants. The ring transfer and needle pass tasks achieved construct validity, had good interrater reliability, and were found to be useful in assessing a novice surgeon's progression, while the EA cut task showed poor reliability.