Text-Guided Image Manipulation via Generative Adversarial Network With Referring Image Segmentation-Based Guidance

This study proposes a novel text-guided image manipulation method that introduces referring image segmentation into a generative adversarial network. The proposed text-guided image manipulation method aims to manipulate images containing multiple objects while preserving text-unrelated regions. The proposed method assigns the task of distinguishing between text-related and unrelated regions in an image to segmentation guidance based on referring image segmentation. With this architecture, the adversarial generative network can focus on generating new attributes according to the text description and reconstructing text-unrelated regions. For the challenging input images with multiple objects, the experimental results demonstrate that the proposed method outperforms conventional methods in terms of image manipulation precision.

Automated summary

This study proposes a novel text-guided image manipulation method by introducing referring image segmentation into a generative adversarial network. The method aims to manipulate images with multiple objects while preserving text-unrelated regions. The proposed method assigns the task of distinguishing between text-related and unrelated regions in an image to segmentation guidance based on referring image segmentation. Experimental results show that the proposed method outperforms conventional methods in terms of image manipulation precision for challenging input images with multiple objects.