Confidence, Accuracy, and Reliability of Penetration-Aspiration Scale Ratings on Flexible Endoscopic Evaluations of Swallowing by Speech Pathologists

This study investigated rater confidence when rating airway invasion with the penetration-aspiration scale (PAS) on flexible endoscopic evaluations of swallowing (FEES), raters' accuracy against a referent-standard, inter-rater reliability, and potential associations between clinician confidence, experience, and accuracy. Thirty-one clinicians who use FEES in their daily practice were asked to judge airway invasion with the PAS and to rate their confidence that their score was correct (0-100) for 40 video clips, five in each of the 8 PAS categories. We found that raters were most confident in rating PAS 1, 7, and 8. The average confidence score across all videos was 76/100. Confidence did not have a significant relationship with accuracy against the referent-standard. Accuracy was highest for PAS 1 (92%), followed by PAS 8 (80%), PAS 7 (77%), and PAS 4 (72%). Accuracy was below 60% for PAS 2, 3, 5, and 6, the lowest being for PAS 3 (49%). Mean accuracy for all ratings, compared to referent-standard ratings, was highest for the intermediate group (71%), followed by expert (68%) and novice (65%). In general, we found that certain PAS scores tend to be rated more accurately, and that participating SLPs had varied confidence in PAS ratings on FEES. Potential reasons for these findings as well as suggested next steps are discussed.

Automated summary

This study investigated rater confidence, accuracy, inter-rater reliability, and potential associations between clinician confidence, experience, and accuracy in evaluating airway invasion using the penetration-aspiration scale (PAS) on flexible endoscopic evaluations of swallowing (FEES). The results showed that certain PAS scores were rated more accurately than others, and there was variability in clinician confidence in PAS ratings. These findings have implications for understanding the factors influencing rater confidence and accuracy in FEES.