Less Finetuning, Better Retrieval: Rethinking LLM Adaptation For Biomedical Retrievers Via Synthetic Data And Model Merging

Abstract

Retrieval-augmented generation (RAG) has become the backbone of grounding Large Language Models (LLMs), improving knowledge updates and reducing hallucinations. Recently, LLM-based retriever models have shown state-of-the-art performance for RAG applications. However, several technical aspects remain underexplored on how to adapt general-purpose LLMs into effective domain-specific retrievers, espe