MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-robust Classifier


  • Mozhdeh Rouhsedaghat University of Southern California
  • Masoud Monajatipoor University of California, Los Angeles
  • C.-C. Jay Kuo University of Southern California
  • Iacopo Masi Sapienza, University of Rome



CV: Applications, CV: Computational Photography, Image & Video Synthesis


We offer a method for one-shot mask-guided image synthesis that allows controlling manipulations of a single image by inverting a quasi-robust classifier equipped with strong regularizers. Our proposed method, entitled MAGIC, leverages structured gradients from a pre-trained quasi-robust classifier to better preserve the input semantics while preserving its classification accuracy, thereby guaranteeing credibility in the synthesis. Unlike current methods that use complex primitives to supervise the process or use attention maps as a weak supervisory signal, MAGIC aggregates gradients over the input, driven by a guide binary mask that enforces a strong, spatial prior. MAGIC implements a series of manipulations with a single framework achieving shape and location control, intense non-rigid shape deformations, and copy/move operations in the presence of repeating objects and gives users firm control over the synthesis by requiring to simply specify binary guide masks. Our study and findings are supported by various qualitative comparisons with the state-of-the-art on the same images sampled from ImageNet and quantitative analysis using machine perception along with a user survey of 100+ participants that endorse our synthesis quality.




How to Cite

Rouhsedaghat, M., Monajatipoor, M., Kuo, C.-C. J., & Masi, I. (2023). MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-robust Classifier. Proceedings of the AAAI Conference on Artificial Intelligence, 37(2), 2172-2179.



AAAI Technical Track on Computer Vision II