Adafuse: Adaptive Diffusion-generated Image And Text Fusion For Interactive Text-to-image Retrieval
2026 · Zhuocheng Zhang, Xingwu Zhang, Kangheng Liang, et al.
Abstract
Recent advances in interactive text-to-image retrieval (I-TIR) use diffusion models to bridge the modality gap between the textual information need and the images to be searched, resulting in increased effectiveness. However, existing frameworks fuse multi-modal views of user feedback by simple embedding addition. In this work, we show that this static and undifferentiated fusion indiscriminately incorporates generative noise produced by the diffusion model, leading to performance degradation for up to 55.62% samples. We further propose ADaFuSE (Adaptive Diffusion-Text Fusion with Semantic-aware Experts), a lightweight fusion model designed to align and calibrate multi-modal views for diffusion-augmented I-TIR, which can be plugged into existing frameworks without modifying the backbone encoder. Specifically, we introduce a dual-branch fusion mechanism that employs an adaptive gating branch to dynamically balance modality reliability, alongside a semantic-aware mixture-of-experts branc
Authors
(none)
Tags
Stats
Related papers
- Eliminating Hallucination In Diffusion-augmented Interactive Text-to-image Retrieval (2026)0.00
- DAFM: Dynamic Adaptive Fusion For Multi-model Collaboration In Composed Image Retrieval (2025)0.00
- Text-guided Synthesis Of Artistic Images With Retrieval-augmented Diffusion Models (2022)8.29
- Detailfusion: A Dual-branch Framework With Detail Enhancement For Composed Image Retrieval (2025)0.00
- Text-to-image Diffusion Models Are Great Sketch-photo Matchmakers (2024)9.41
- Category-level Text-to-image Retrieval Improved: Bridging The Domain Gap With Diffusion Models And Vision Encoders (2025)1.20
- Far-net: Multi-stage Fusion Network With Enhanced Semantic Alignment And Adaptive Reconciliation For Composed Image Retrieval (2025)0.00
- Image Retrieval Outperforms Diffusion Models On Data Augmentation (2023)0.00