Align Then Train: Efficient Retrieval Adapter Learning

Abstract

Dense retrieval systems increasingly need to handle complex queries. In many realistic settings, users express intent through long instructions or task-specific descriptions, while target documents remain relatively simple and static. This asymmetry creates a retrieval mismatch: understanding queries may require strong reasoning and instruction-following, whereas efficient document indexing favors lightweight encoders. Existing retrieval systems often address this mismatch by directly improving the embedding model, but fine-tuning large embedding models to better follow such instructions is computationally expensive, memory-intensive, and operationally burdensome. To address this challenge, we propose Efficient Retrieval Adapter (ERA), a label-efficient framework that trains retrieval adapters in two stages: self-supervised alignment and supervised adaptation. Inspired by the pre-training and supervised fine-tuning stages of LLMs, ERA first aligns the embedding spaces of a large query

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymaekawa2026align

Related papers

• Learning To Select: Query-aware Adaptive Dimension Selection For Dense Retrieval (2026)0.00
• Federated Learning With Ad-hoc Adapter Insertions: The Case Of Soft-embeddings For Training Classifier-as-retriever (2025)0.00
• Expandr: Teaching Dense Retrievers Beyond Queries With LLM Guidance (2025)3.25
• PEFA: Parameter-free Adapters For Large-scale Embedding-based Retrieval Models (2023)7.73
• Parameter-efficient Sparse Retrievers And Rerankers Using Adapters (2023)4.52
• Search-adaptor: Embedding Customization For Information Retrieval (2023)0.00
• Large Reasoning Embedding Models: Towards Next-generation Dense Retrieval Paradigm (2025)0.00
• Query Encoder Distillation Via Embedding Alignment Is A Strong Baseline Method To Boost Dense Retriever Online Efficiency (2023)0.00