Eliminating Hallucination In Diffusion-augmented Interactive Text-to-image Retrieval

Abstract

Diffusion-Augmented Interactive Text-to-Image Retrieval (DAI-TIR) is a promising paradigm that improves retrieval performance by generating query images via diffusion models and using them as additional ``views'' of the user's intent. However, these generative views can be incorrect because diffusion generation may introduce hallucinated visual cues that conflict with the original query text. Indeed, we empirically demonstrate that these hallucinated cues can substantially degrade DAI-TIR performance. To address this, we propose Diffusion-aware Multi-view Contrastive Learning (DMCL), a hallucination-robust training framework that casts DAI-TIR as joint optimization over representations of query intent and the target image. DMCL introduces semantic-consistency and diffusion-aware contrastive objectives to align textual and diffusion-generated query views while suppressing hallucinated query signals. This yields an encoder that acts as a semantic filter, effectively mapping hallucinated

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2026eliminating

Related papers

• Adafuse: Adaptive Diffusion-generated Image And Text Fusion For Interactive Text-to-image Retrieval (2026)0.00
• Text-guided Synthesis Of Artistic Images With Retrieval-augmented Diffusion Models (2022)8.29
• MV-RAG: Retrieval Augmented Multiview Diffusion (2025)0.00
• Daug: Diffusion-based Channel Augmentation For Radiology Image Retrieval And Classification (2024)0.00
• Category-level Text-to-image Retrieval Improved: Bridging The Domain Gap With Diffusion Models And Vision Encoders (2025)1.20
• Dynamic Contrastive Distillation For Image-text Retrieval (2022)11.76
• Alleviating Hallucination In Large Vision-language Models With Active Retrieval Augmentation (2024)7.16
• DAFM: Dynamic Adaptive Fusion For Multi-model Collaboration In Composed Image Retrieval (2025)0.00