Neighborretr: Balancing Hub Centrality In Cross-modal Retrieval

Abstract

Cross-modal retrieval aims to bridge the semantic gap between different modalities, such as visual and textual data, enabling accurate retrieval across them. Despite significant advancements with models like CLIP that align cross-modal representations, a persistent challenge remains: the hubness problem, where a small subset of samples (hubs) dominate as nearest neighbors, leading to biased representations and degraded retrieval accuracy. Existing methods often mitigate hubness through post-hoc normalization techniques, relying on prior data distributions that may not be practical in real-world scenarios. In this paper, we directly mitigate hubness during training and introduce NeighborRetr, a novel method that effectively balances the learning of hubs and adaptively adjusts the relations of various kinds of neighbors. Our approach not only mitigates the hubness problem but also enhances retrieval performance, achieving state-of-the-art results on multiple cross-modal retrieval benchma

Stats

• citations1
• S2 citations—
• github stars8
• HF likes0
• heat score4.17
• arxiv keylin2025neighborretr

Related papers

• Hubness Reduction With Dual Bank Sinkhorn Normalization For Cross-modal Retrieval (2025)0.95
• Balance Act: Mitigating Hubness In Cross-modal Retrieval With Query And Gallery Banks (2023)8.46
• Adversarial Hubness In Multi-modal Retrieval (2024)0.00
• Cross Modal Retrieval With Querybank Normalisation (2021)14.06
• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• Cross-modal Center Loss (2020)0.00
• Neighbor-aware Instance Refining With Noisy Labels For Cross-modal Retrieval (2025)2.26
• Continual Learning In Cross-modal Retrieval (2021)9.41