Intraoperative robotic-assisted low anterior rectal resection performance assessment using procedure-specific binary metrics and a global rating scale

In this prospective, multicentre study, we objectively assessed the performance of experienced and novice colorectal surgeons performing robotic-assisted low anterior rectal resection (for rectal cancer surgery) with two assessment methods: proficient-based progressive (PBP), and GEARS. Both assessment methods discriminated between group performances, but PBP binary metrics seem to be a more useful tool in this study. PBP binary metrics revealed that some experienced surgeons performed poorly. Background This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure. Method A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (<= 30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated. Results Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 - Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019-0.008; GEARS scores, P = 0.029-0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15-41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58-64 per cent). Conclusions PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.

Automated summary

In this study, the performance of experienced and novice colorectal surgeons in robotic-assisted low anterior rectal resection (RA-LAR) was objectively assessed using two assessment methods: proficiency-based progression (PBP) and global evaluative assessment of robotic skills (GEARS). Both assessment methods were able to distinguish between the performances of the two groups, but the PBP binary metrics were found to be more useful. The study showed that some experienced surgeons performed poorly, as revealed by the PBP binary metrics.