SEGMENTATION-AWARE TEXT-GUIDED IMAGE MANIPULATION

We propose a novel approach that improves text-guided image manipulation performance in this paper. Text-guided image manipulation aims at modifying some parts of an input image in accordance with the user's text description by semantically associating the regions of the image with the text description. We tackle the conventional methods' problem of modifying undesired parts caused by differences in representation ability between text descriptions and images. Humans tend to pay attention primarily to objects corresponding to the foreground of images, and text descriptions by humans mostly represent the foreground. Therefore, it is necessary to introduce not only a foreground-aware bias based on text descriptions but also a background-aware bias that the text descriptions do not represent. We introduce an image segmentation network into the generative adversarial network for image manipulation to solve the above problem. Comparative experiments with three state-of-the-art methods show the effectiveness of our method quantitatively and qualitatively.

Automated summary

This paper presents a novel approach to improve text-guided image manipulation performance by introducing foreground-aware and background-aware biases, addressing the issue of modifying undesired parts in images caused by differences in representation ability between text descriptions and images. By integrating an image segmentation network into the generative adversarial network for image manipulation, the effectiveness of the proposed method is demonstrated through comparative experiments with three state-of-the-art methods.