U-Turn: Crafting Adversarial Queries with Opposite-Direction Features

This paper aims to craft adversarial queries for image retrieval, which uses image features for similarity measurement. Many commonly used methods are developed in the context of image classification. However, these methods, which attack prediction probabilities, only exert an indirect influence on the image features and are thus found less effective when being applied to the retrieval problem. In designing an attack method specifically for image retrieval, we introduce opposite-direction feature attack (ODFA), a white-box attack approach that directly attacks query image features to generate adversarial queries. As the name implies, the main idea underpinning ODFA is to impel the original image feature to the opposite direction, similar to a U-turn. This simple idea is experimentally evaluated on five retrieval datasets. We show that the adversarial queries generated by ODFA cause true matches no longer to be seen at the top ranks, and the attack success rate is consistently higher than classifier attack methods. In addition, our method of creating adversarial queries can be extended for multi-scale query inputs and is generalizable to other retrieval models without foreknowing their weights, i.e., the black-box setting.

Automated summary

This paper aims to generate adversarial queries for image retrieval by directly attacking query image features. The proposed opposite-direction feature attack (ODFA) method induces the original image feature in the opposite direction to create adversarial queries. Experimental results on five retrieval datasets demonstrate that ODFA outperforms classifier attack methods in terms of attack success rate, causing true matches to be less visible in the top ranks. Moreover, the method is extendable to multi-scale query inputs and applicable in black-box settings.