SpecSeg Network for Specular Highlight Detection and Segmentation in Real-World Images

Specular highlights detection and removal in images is a fundamental yet non-trivial problem of interest. Most modern techniques proposed are inadequate at dealing with real-world images taken under uncontrolled conditions with the presence of complex textures, multiple objects, and bright colours, resulting in reduced accuracy and false positives. To detect specular pixels in a wide variety of real-world images independent of the number, colour, or type of illuminating source, we propose an efficient Specular Segmentation (SpecSeg) network based on the U-net architecture that is expeditious to train on nominal-sized datasets. The proposed network can detect pixels strongly affected by specular highlights with a high degree of precision, as shown by comparison with the state-of-the-art methods. The technique proposed is trained on publicly available datasets and tested using a large selection of real-world images with highly encouraging results.

Automated summary

Detecting and removing specular highlights in images is a significant problem. Existing techniques are inadequate for real-world images with complex textures and multiple objects. This paper proposes an efficient Specular Segmentation network based on the U-net architecture, which can accurately detect specular pixels in various real-world images.