ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing
Keywords:CV: Computational Photography, Image & Video Synthesis
AbstractThe StyleGAN family succeed in high-fidelity image generation and allow for flexible and plausible editing of generated images by manipulating the semantic-rich latent style space. However, projecting a real image into its latent space encounters an inherent trade-off between inversion quality and editability. Existing encoder-based or optimization-based StyleGAN inversion methods attempt to mitigate the trade-off but see limited performance. To fundamentally resolve this problem, we propose a novel two-phase framework by designating two separate networks to tackle editing and reconstruction respectively, instead of balancing the two. Specifically, in Phase I, a W-space-oriented StyleGAN inversion network is trained and used to perform image inversion and edit- ing, which assures the editability but sacrifices reconstruction quality. In Phase II, a carefully designed rectifying network is utilized to rectify the inversion errors and perform ideal reconstruction. Experimental results show that our approach yields near-perfect reconstructions without sacrificing the editability, thus allowing accurate manipulation of real images. Further, we evaluate the performance of our rectifying net- work, and see great generalizability towards unseen manipulation types and out-of-domain images.
How to Cite
Li, B., Ma, T., Zhang, P., Hua, M., Liu, W., He, Q., & Yi, Z. (2023). ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing. Proceedings of the AAAI Conference on Artificial Intelligence, 37(1), 1269-1277. https://doi.org/10.1609/aaai.v37i1.25210
AAAI Technical Track on Computer Vision I