Multi-observer contouring of male pelvic anatomy: Highly variable agreement across conventional and emerging structures of interest

Introduction This study quantified inter-observer contouring variations for multiple male pelvic structures, many of which are of emerging relevance for prostate cancer radiotherapy progression and toxicity response studies. Methods Five prostate cancer patient datasets (CT and T2-weighted MR) were distributed to 13 observers for contouring. CT structures contoured included the clinical target volume (CTV), seminal vesicles, rectum, colon, bowel bag, bladder and peri-rectal space (PRS). MR contours included CTV, trigone, membranous urethra, penile bulb, neurovascular bundle and multiple pelvic floor muscles. Contouring variations were assessed using the intraclass correlation coefficient (ICC), Dice similarity coefficient (DSC), and multiple additional metrics. Results Clinical target volume (CT and MR), bladder, rectum and PRS contours showed excellent inter-observer agreement (median ICC = 0.97; 0.99; 1.00; 0.95; 0.90, DSC = 0.83 +/- 0.05; 0.88 +/- 0.05; 0.93 +/- 0.03; 0.81 +/- 0.07; 0.80 +/- 0.06, respectively). Seminal vesicle contours were more variable (ICC = 0.75, DSC = 0.73 +/- 0.14), while colon and bowel bag contoured volumes were consistent (ICC = 0.97; 0.97), but displayed poor overlap (DSC = 0.58 +/- 0.22; 0.67 +/- 0.21). Smaller MR structures showed significant inter-observer variations, with poor overlap for trigone, membranous urethra, penile bulb, and left and right neurovascular bundles (DSC = 0.44 +/- 0.22; 0.41 +/- 0.21; 0.66 +/- 0.21; 0.16 +/- 0.17; 0.15 +/- 0.15). Pelvic floor muscles recorded moderate to strong inter-observer agreement (ICC = 0.50-0.97), although large outlier variations were observed. Conclusions Inter-observer contouring variation was significant for multiple pelvic structures contoured on MR.