Retrievegan: Image Synthesis Via Differentiable Patch Retrieval

Abstract

Image generation from scene description is a cornerstone technique for the controlled generation, which is beneficial to applications such as content creation and image editing. In this work, we aim to synthesize images from scene description with retrieved patches as reference. We propose a differentiable retrieval module. With the differentiable retrieval module, we can (1) make the entire pipeline end-to-end trainable, enabling the learning of better feature embedding for retrieval; (2) encourage the selection of mutually compatible patches with additional objective functions. We conduct extensive quantitative and qualitative experiments to demonstrate that the proposed method can generate realistic and diverse images, where the retrieved patches are reasonable and mutually compatible.

Stats

• citations20
• S2 citations—
• github stars0
• HF likes0
• heat score9.92
• arxiv keytseng2020retrievegan

Related papers

• AR-RAG: Autoregressive Retrieval Augmentation For Image Generation (2025)0.00
• Irgen: Generative Modeling For Image Retrieval (2023)7.16
• Retrieval-guided Cross-view Image Synthesis (2024)0.00
• Text-guided Synthesis Of Artistic Images With Retrieval-augmented Diffusion Models (2022)8.29
• Semi-parametric Neural Image Synthesis (2022)0.00
• Imagerag: Dynamic Image Retrieval For Reference-guided Image Generation (2025)0.00
• Patch-wise Retrieval: A Bag Of Practical Techniques For Instance-level Matching (2025)0.00
• Drop The GAN: In Defense Of Patches Nearest Neighbors As Single Image Generative Models (2021)12.87