Refir: Grounding Large Restoration Models With Retrieval Augmentation

Abstract

Recent advances in diffusion-based Large Restoration Models (LRMs) have significantly improved photo-realistic image restoration by leveraging the internal knowledge embedded within model weights. However, existing LRMs often suffer from the hallucination dilemma, i.e., producing incorrect contents or textures when dealing with severe degradations, due to their heavy reliance on limited internal knowledge. In this paper, we propose an orthogonal solution called the Retrieval-augmented Framework for Image Restoration (ReFIR), which incorporates retrieved images as external knowledge to extend the knowledge boundary of existing LRMs in generating details faithful to the original scene. Specifically, we first introduce the nearest neighbor lookup to retrieve content-relevant high-quality images as reference, after which we propose the cross-image injection to modify existing LRMs to utilize high-quality textures from retrieved images. Thanks to the additional external knowledge, our ReFIR

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2024refir

Related papers

• RASR: Retrieval-augmented Super Resolution For Practical Reference-based Image Restoration (2025)0.00
• Alleviating Hallucination In Large Vision-language Models With Active Retrieval Augmentation (2024)7.16
• Realrag: Retrieval-augmented Realistic Image Generation Via Self-reflective Contrastive Learning (2025)0.00
• Adversarial Reconstruction Feedback For Robust Fine-grained Generalization (2025)0.00
• Air-know: Arbiter-calibrated Knowledge-internalizing Robust Network For Composed Image Retrieval (2026)0.00
• Text-guided Synthesis Of Artistic Images With Retrieval-augmented Diffusion Models (2022)8.29
• Retrieval-augmented Perception: High-resolution Image Perception Meets Visual RAG (2025)0.00
• Enhancing Image Quality Assessment Ability Of Lmms Via Retrieval-augmented Generation (2026)0.00