SDE-RAE:CLIP-based realistic image reconstruction and editing network using stochastic differential diffusion

Summary: This study introduces a text-driven method that allows training a generative model to produce images from different domains guided by text prompts and without seeing any image. The method demonstrates its effectiveness in adapting the generator to various domains with diverse styles and shapes, which would be difficult to achieve with existing methods.

Summary: This paper studies the latent space of the StyleGAN unconditional generator and identifies tradeoffs between distortion-editability and distortion-perception within the latent space. Two principles for designing encoders to control the proximity of inversions to the regions originally trained on StyleGAN are suggested. By qualitatively and quantitatively evaluating on challenging domains like cars and horses, the inversion method presented here, along with common editing techniques, achieves superior real-image editing quality with only a minor drop in reconstruction accuracy.