Cross Modal Retrieval With Querybank Normalisation
2021 · Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin, et al.
Abstract
Profiting from large-scale training datasets, advances in neural architecture design and efficient inference, joint embeddings have become the dominant approach for tackling cross-modal retrieval. In this work we first show that, despite their effectiveness, state-of-the-art joint embeddings suffer significantly from the longstanding "hubness problem" in which a small number of gallery embeddings form the nearest neighbours of many queries. Drawing inspiration from the NLP literature, we formulate a simple but effective framework called Querybank Normalisation (QB-Norm) that re-normalises query similarities to account for hubs in the embedding space. QB-Norm improves retrieval performance without requiring retraining. Differently from prior work, we show that QB-Norm works effectively without concurrent access to any test set queries. Within the QB-Norm framework, we also propose a novel similarity normalisation method, the Dynamic Inverted Softmax, that is significantly more robust th
Authors
(none)
Tags
Stats
Related papers
- Balance Act: Mitigating Hubness In Cross-modal Retrieval With Query And Gallery Banks (2023)8.46
- Neighborretr: Balancing Hub Centrality In Cross-modal Retrieval (2025)4.17
- Hubness Reduction With Dual Bank Sinkhorn Normalization For Cross-modal Retrieval (2025)0.95
- Universal Vision-language Dense Retrieval: Learning A Unified Representation Space For Multi-modal Retrieval (2022)3.45
- Cross-modal Retrieval Augmentation For Multi-modal Classification (2021)9.23
- Retrieve Fast, Rerank Smart: Cooperative And Joint Approaches For Improved Cross-modal Retrieval (2021)10.97
- Webly Supervised Joint Embedding For Cross-modal Image-text Retrieval (2018)13.17
- Joint Fusion And Encoding: Advancing Multimodal Retrieval From The Ground Up (2025)0.00