PEFA: Parameter-free Adapters For Large-scale Embedding-based Retrieval Models

Abstract

Embedding-based Retrieval Models (ERMs) have emerged as a promising framework for large-scale text retrieval problems due to powerful large language models. Nevertheless, fine-tuning ERMs to reach state-of-the-art results can be expensive due to the extreme scale of data as well as the complexity of multi-stages pipelines (e.g., pre-training, fine-tuning, distillation). In this work, we propose the PEFA framework, namely ParamEter-Free Adapters, for fast tuning of ERMs without any backward pass in the optimization. At index building stage, PEFA equips the ERM with a non-parametric k-nearest neighbor (kNN) component. At inference stage, PEFA performs a convex combination of two scoring functions, one from the ERM and the other from the kNN. Based on the neighborhood definition, PEFA framework induces two realizations, namely PEFA-XL (i.e., extra large) using double ANN indices and PEFA-XS (i.e., extra small) using a single ANN index. Empirically, PEFA achieves significant improvement on

Stats

• citations1
• S2 citations—
• github stars542
• HF likes0
• heat score7.73
• arxiv keychang2023pefa

Related papers

• Align Then Train: Efficient Retrieval Adapter Learning (2026)0.00
• Plug-and-play Parameter-efficient Tuning Of Embeddings For Federated Recommendation (2025)0.95
• Pebr: A Probabilistic Approach To Embedding Based Retrieval (2024)0.00
• Parameter-efficient Sparse Retrievers And Rerankers Using Adapters (2023)4.52
• Refining Joint Text And Source Code Embeddings For Retrieval Task With Parameter-efficient Fine-tuning (2024)0.00
• Search-adaptor: Embedding Customization For Information Retrieval (2023)0.00
• Progressively Optimized Bi-granular Document Representation For Scalable Embedding Based Retrieval (2022)11.06
• Parameter-efficient Neural Reranking For Cross-lingual And Multilingual Retrieval (2022)0.00