Comparative validity of three simulation platforms for objective assessment of otoscopy skills

Objectives To assess the face, construct and content validity of three different platforms for otoscopy skills assessment, using a traditional otoscope with manikin, digital otoscope (Tympahealth) with manikin, and traditional otoscope with a low-cost model ear (SimEar).Design Prospective mixed methods study.Setting Tertiary hospital.Participants Postgraduate trainees and expert assessors.Main Outcome Measures Face and Content validity based on expert assessor ranking on each model and their feedback from semi-structured interviews. Construct validity based on Objective Structured Clinical Examination scores.Results Each platform differed in face, construct and content validity scores, with no one platform consistently outperforming others. Three main themes were identified during thematic analysis of expert assessor interviews: ability to assess what is seen, anatomical reality, and ease of use. The low-cost model showed greatest potential, where modification to include a silicone ear could lead to high validity with marginal increase in cost.Conclusion Several modalities for assessing otoscopy skills exist, each with advantages and disadvantages. Modifications to a low-cost model, for use with either a traditional or digital otoscope, could prove to be the best model.

Automated summary

This study assessed the face, construct, and content validity of three different platforms for otoscopy skills assessment. The platforms included a traditional otoscope with a manikin, a digital otoscope with a manikin, and a traditional otoscope with a low-cost model ear. The results showed that each platform had different validity scores and there was no consistent best performing platform. Thematic analysis of expert assessor interviews identified three main themes: ability to assess what is seen, anatomical reality, and ease of use. The low-cost model showed the greatest potential with modifications to include a silicone ear.