An Empirical Study Of Position Bias In Modern Information Retrieval

Abstract

This study investigates the position bias in information retrieval, where models tend to overemphasize content at the beginning of passages while neglecting semantically relevant information that appears later. To analyze the extent and impact of position bias, we introduce a new evaluation framework consisting of two position-aware retrieval benchmarks (SQuAD-PosQ, FineWeb-PosQ) and an intuitive diagnostic metric, the Position Sensitivity Index (PSI), for quantifying position bias from a worst-case perspective. We conduct a comprehensive evaluation across the full retrieval pipeline, including BM25, dense embedding models, ColBERT-style late-interaction models, and full-interaction reranker models. Our experiments show that when relevant information appears later in the passage, dense embedding models and ColBERT-style models suffer significant performance degradation (an average drop of 15.6%). In contrast, BM25 and reranker models demonstrate greater robustness to such positional va

Stats

• citations0
• S2 citations—
• github stars6
• HF likes0
• heat score1.69
• arxiv keyzeng2025an

Related papers

• Posir: Position-aware Heterogeneous Information Retrieval Benchmark (2026)0.00
• Quantifying Positional Biases In Text Embedding Models (2024)0.00
• Positional Bias In Multimodal Embedding Models: Do They Favor The Beginning, The Middle, Or The End? (2025)0.00
• Do Neural Ranking Models Intensify Gender Bias? (2020)12.47
• Debiasing Gender Bias In Information Retrieval Models (2022)0.00
• Writing Style Matters: An Examination Of Bias And Fairness In Information Retrieval Systems (2024)4.52
• Mitigating Test-time Bias For Fair Image Retrieval (2023)0.00
• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58