Full-Reference Image Quality Assessment: Addressing Content Misalignment Issue by Comparing Order Statistics of Deep Features

This letter aims to develop advanced full-reference image quality assessment (FR-IQA) models to evaluate content-misaligned image pairs, which are commonly encountered in image reconstruction tasks and texture synthesis tasks. Traditional FR-IQA models tend to be overly sensitive to content shifting and misalignment, thus deviating from subjective evaluations. Herein, we propose a deep order statistical similarity (DOSS) FR-IQA model that compares the order statistics of deep features to address this issue. In DOSS, the reference and distorted images are projected into the deep feature space, and the sorted deep network features are compared with the cosine similarity index to output the final perceptual quality scores. With such a simple design baseline, DOSS offers several advantages. First, it mimics the behavior of the human visual system (HVS) in terms of evaluating content-misaligned image pairs, thereby tolerating slight image shifts and deformations. Second, DOSS possesses an advanced texture perception capability, producing superior quality assessment results on images generated by various texture synthesis algorithms; this indicates that DOSS can be used to select visually appealing texture synthesis results. Finally, experimental results demonstrate that DOSS can also obtain competitive quality assessment results on standard IQA datasets, suggesting that deep feature order statistics can serve as generic features for both content-aligned and content-misaligned IQA. The code for this method is publicly available at https://github.com/Buka-Xing/DOSS.

Automated summary

This paper proposes a deep order statistical similarity (DOSS) FR-IQA model to evaluate content-misaligned image pairs encountered in image reconstruction and texture synthesis tasks. DOSS compares the order statistics of deep features in the reference and distorted images to output perceptual quality scores. It mimics the human visual system's behavior and possesses advanced texture perception capability, producing superior quality assessment results on various texture synthesis algorithms.