Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physicianreported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning. CO 2021 Elsevier B.V. All rights reserved. Radiotherapy and Oncology 160 (2021) 185-191

Automated summary

Advances in AI-based methods have led to the development of auto-segmentation systems in radiotherapy. However, there is no uniform standard for evaluating their efficacy. Common evaluation techniques include geometric overlap, dosimetric parameters, and clinical rating scales, with many geometric indices showing weak correlation with clinical endpoints. A multi-domain evaluation is necessary to assess the clinical readiness of auto-segmentation for radiation treatment planning.