Parameter-efficient Sparse Retrievers And Rerankers Using Adapters

Abstract

Parameter-Efficient transfer learning with Adapters have been studied in Natural Language Processing (NLP) as an alternative to full fine-tuning. Adapters are memory-efficient and scale well with downstream tasks by training small bottle-neck layers added between transformer layers while keeping the large pretrained language model (PLMs) frozen. In spite of showing promising results in NLP, these methods are under-explored in Information Retrieval. While previous studies have only experimented with dense retriever or in a cross lingual retrieval scenario, in this paper we aim to complete the picture on the use of adapters in IR. First, we study adapters for SPLADE, a sparse retriever, for which adapters not only retain the efficiency and effectiveness otherwise achieved by finetuning, but are memory-efficient and orders of magnitude lighter to train. We observe that Adapters-SPLADE not only optimizes just 2% of training parameters, but outperforms fully fine-tuned counterpart and exist

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keypal2023parameter

Related papers

• Parameter-efficient Neural Reranking For Cross-lingual And Multilingual Retrieval (2022)0.00
• Cross-modal Adapter: Parameter-efficient Transfer Learning Approach For Vision-language Models (2024)6.77
• Federated Learning With Ad-hoc Adapter Insertions: The Case Of Soft-embeddings For Training Classifier-as-retriever (2025)0.00
• RRRA: Resampling And Reranking Through A Retriever Adapter (2025)0.00
• Align Then Train: Efficient Retrieval Adapter Learning (2026)0.00
• Uniadapter: Unified Parameter-efficient Transfer Learning For Cross-modal Modeling (2023)3.77
• PEFA: Parameter-free Adapters For Large-scale Embedding-based Retrieval Models (2023)7.73
• Search-adaptor: Embedding Customization For Information Retrieval (2023)0.00