Integrating Listwise Ranking Into Pairwise-based Image-text Retrieval

Abstract

Image-Text Retrieval (ITR) is essentially a ranking problem. Given a query caption, the goal is to rank candidate images by relevance, from large to small. The current ITR datasets are constructed in a pairwise manner. Image-text pairs are annotated as positive or negative. Correspondingly, ITR models mainly use pairwise losses, such as triplet loss, to learn to rank. Pairwise-based ITR increases positive pair similarity while decreasing negative pair similarity indiscriminately. However, the relevance between dissimilar negative pairs is different. Pairwise annotations cannot reflect this difference in relevance. In the current datasets, pairwise annotations miss many correlations. There are many potential positive pairs among the pairs labeled as negative. Pairwise-based ITR can only rank positive samples before negative samples, but cannot rank negative samples by relevance. In this paper, we integrate listwise ranking into conventional pairwise-based ITR. Listwise ranking optimizes

Stats

• citations8
• S2 citations—
• github stars9
• HF likes0
• heat score9.16
• arxiv keyli2023integrating

Related papers

• Active Learning For Finely-categorized Image-text Retrieval By Selecting Hard Negative Unpaired Samples (2024)2.26
• Lexlip: Lexicon-bottlenecked Language-image Pre-training For Large-scale Image-text Retrieval (2023)10.85
• Image-text Retrieval: A Survey On Recent Research And Development (2022)13.93
• Learnable Pillar-based Re-ranking For Image-text Retrieval (2023)9.92
• When Vision Meets Texts In Listwise Reranking (2026)0.00
• Ranking-aware Uncertainty For Text-guided Image Retrieval (2023)0.00
• Chain-of-thought Re-ranking For Image Retrieval Tasks (2025)1.81
• CODER: Coupled Diversity-sensitive Momentum Contrastive Learning For Image-text Retrieval (2022)13.72