Deep saliency models : The quest for the loss function

Deep learning techniques are widely used to model human visual saliency, to such a point that state-ofthe-art performances are now only attained by deep neural networks. However, one key part of a typical deep learning model is often neglected when it comes to modeling visual saliency: the choice of the loss function. In this work, we explore some of the most popular loss functions that are used in deep saliency models. We demonstrate that on a fixed network architecture, modifying the loss function can significantly improve (or depreciate) the results, hence emphasizing the importance of the choice of the loss function when designing a model. We also evaluate the relevance of new loss functions for saliency prediction inspired by metrics used in style-transfer tasks. Finally, we show that a linear combination of several well-chosen loss functions leads to significant improvements in performance on different datasets as well as on a different network architecture, thus demonstrating the robustness of a combined metric. CO 2020 Elsevier B.V. All rights reserved.

Automated summary

Deep learning techniques are widely used for modeling human visual saliency, with the choice of the loss function being a key factor that can significantly impact results. This study demonstrates that modifying the loss function on a fixed network architecture can lead to improvements or depreciation in performance. Combining several well-chosen loss functions in a linear combination can lead to significant improvements in performance on different datasets and network architectures.