Generalization of the shortest path approach for superpixel segmentation of omnidirectional images

With the growing use of image capture devices using wide angles and the need for fast and accurate image analysis in computer vision, there is a demand for dedicated under-representation methods. Most decomposition methods segment an image into a small number of irregular homogeneous regions, called superpixels. Nevertheless, these approaches are generally designed to process natural 2D planar images, i.e., captured with a 90 o angle view without distortion. In this work, we present SphSPS, a new gen-eral decomposition method (for Spherical Shortest Path-based Superpixels)1, that is dedicated to wide 360 o omnidirectional or spherical images. The produced superpixels respect both the geometry of the 3D spherical acquisition space, and the boundaries of image objects. To fastly extract relevant clustering features, we generalize the shortest path approach between a pixel and a superpixel center. We demon-strate that considering the geometry of the acquisition space to compute the shortest path enables to jointly improve the segmentation accuracy and the shape regularity of superpixels. To evaluate this reg-ularity property, we propose a generalization of a standard 2D regularity metric to the spherical space, addressing the limitations of the only existing spherical compactness measure. Finally, SphSPS is vali-dated on reference 360 o images from the PSD (Panorama Segmentation Dataset) and also on synthetic road omnidirectional images. Our method significantly outperforms both planar and spherical state-of -the-art approaches in terms of segmentation accuracy, robustness to noise and regularity, providing a very interesting tool for superpixel-based applications on 360 o images.(c) 2023 Elsevier Ltd. All rights reserved.

Automated summary

With the increasing use of wide-angle image capture devices and the need for fast and accurate image analysis in computer vision, there is a demand for dedicated under-representation methods. This work presents SphSPS, a new decomposition method for wide 360° omnidirectional or spherical images, which respects both the geometry of the 3D spherical acquisition space and the boundaries of image objects. It significantly outperforms existing methods in terms of segmentation accuracy, robustness to noise, and regularity, providing a valuable tool for superpixel-based applications on 360° images.