CEQE: Contextualized Embeddings For Query Expansion

Abstract

In this work we leverage recent advances in context-sensitive language models to improve the task of query expansion. Contextualized word representation models, such as ELMo and BERT, are rapidly replacing static embedding models. We propose a new model, Contextualized Embeddings for Query Expansion (CEQE), that utilizes query-focused contextualized embedding vectors. We study the behavior of contextual representations generated for query expansion in ad-hoc document retrieval. We conduct our experiments on probabilistic retrieval models as well as in combination with neural ranking models. We evaluate CEQE on two standard TREC collections: Robust and Deep Learning. We find that CEQE outperforms static embedding-based expansion methods on multiple collections (by up to 18% on Robust and 31% on Deep Learning on average precision) and also improves over proven probabilistic pseudo-relevance feedback (PRF) models. We further find that multiple passes of expansion and reranking result in c

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keynaseri2021ceqe

Related papers

• Deep Neural Networks For Query Expansion Using Word Embeddings (2018)11.58
• On The Interpolation Of Contextualized Term-based Ranking With BM25 For Query-by-example Retrieval (2022)7.50
• Query Expansion With Locally-trained Word Embeddings (2016)16.14
• Hyqe: Ranking Contexts With Hypothetical Query Embeddings (2024)5.74
• Contextualized Query Embeddings For Conversational Search (2021)10.21
• Expandr: Teaching Dense Retrievers Beyond Queries With LLM Guidance (2025)3.25
• CODER: An Efficient Framework For Improving Retrieval Through Contextual Document Embedding Reranking (2021)7.16
• Llm-augmented Retrieval: Enhancing Retrieval Models Through Language Models And Doc-level Embedding (2024)0.00