Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector

Accurate segmentation of the prostate and organs at risk in computed tomography (CT) images is a crucial step for radiotherapy planning. Manual segmentation, as performed nowadays, is a time consuming process and prone to errors due to the a high intra-and inter-expert variability. This paper introduces a new automatic method for prostate, rectum and bladder segmentation in planning CT using a geometrical shape model under a Bayesian framework. A set of prior organ shapes are first built by applying principal component analysis to a population of manually delineated CT images. Then, for a given individual, the most similar shape is obtained by mapping a set of multi-scale edge observations to the space of organs with a customized likelihood function. Finally, the selected shape is locally deformed to adjust the edges of each organ. Experiments were performed with real data from a population of 116 patients treated for prostate cancer. The data set was split in training and test groups, with 30 and 86 patients, respectively. Results show that the method produces competitive segmentations w.r.t standard methods (averaged dice = 0.91 for prostate, 0.94 for bladder, 0.89 for rectum) and outperforms the majority-vote multi-atlas approaches (using rigid registration, free-form deformation and the demons algorithm).