Language-Guided Global Image Editing via Cross-Modal Cyclic Mechanism

Editing an image automatically via a linguistic request can significantly save laborious manual work and is friendly to photography novice. In this paper, we focus on the task of language-guided global image editing. Existing works suffer from imbalanced and insufficient data distribution of real-world datasets and thus fail to understand language requests well. To handle this issue, we propose to create a cycle with our image generator by creating a novel model called Editing Description Network (EDNet) which predicts an editing embedding given a pair of images. Given the cycle, we propose several free augmentation strategies to help our model understand various editing requests given the imbalanced dataset. In addition, two other novel ideas are proposed: an Image-Request Attention (IRA) module which allows our method to edit an image spatial-adaptively when the image requires different editing degree at different regions, as well as a new evaluation metric for this task which is more semantic and reasonable than conventional pixel losses (e.g. L1). Extensive experiments on two benchmark datasets demonstrate the effectiveness of our method over existing approaches.

Automated summary

This paper focuses on the task of language-guided global image editing, proposing a method that creates a cycle with an image generator and utilizes the Editing Description Network (EDNet) model to predict editing embeddings. By introducing free augmentation strategies, the Image-Request Attention (IRA) module, and a new evaluation metric, it successfully addresses the issues of understanding language requests and handling imbalanced datasets present in existing approaches.