Semi-parametric Neural Image Synthesis

Abstract

Novel architectures have recently improved generative image synthesis leading to excellent visual quality in various tasks. Much of this success is due to the scalability of these architectures and hence caused by a dramatic increase in model complexity and in the computational resources invested in training these models. Our work questions the underlying paradigm of compressing large training data into ever growing parametric representations. We rather present an orthogonal, semi-parametric approach. We complement comparably small diffusion or autoregressive models with a separate image database and a retrieval strategy. During training we retrieve a set of nearest neighbors from this external database for each training instance and condition the generative model on these informative samples. While the retrieval approach is providing the (local) content, the model is focusing on learning the composition of scenes based on this content. As demonstrated by our experiments, simply swappi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyblattmann2022semi

Related papers

• Text-guided Synthesis Of Artistic Images With Retrieval-augmented Diffusion Models (2022)8.29
• Retrievegan: Image Synthesis Via Differentiable Patch Retrieval (2020)9.92
• Retrieval-guided Cross-view Image Synthesis (2024)0.00
• Drop The GAN: In Defense Of Patches Nearest Neighbors As Single Image Generative Models (2021)12.87
• Realrag: Retrieval-augmented Realistic Image Generation Via Self-reflective Contrastive Learning (2025)0.00
• Generative Model For Zero-shot Sketch-based Image Retrieval (2019)9.23
• AR-RAG: Autoregressive Retrieval Augmentation For Image Generation (2025)0.00
• Text-to-image Generation Via Implicit Visual Guidance And Hypernetwork (2022)0.00