Retro-li: Small-scale Retrieval Augmented Generation Supporting Noisy Similarity Searches And Domain Shift Generalization

Abstract

The retrieval augmented generation (RAG) system such as Retro has been shown to improve language modeling capabilities and reduce toxicity and hallucinations by retrieving from a database of non-parametric memory containing trillions of entries. We introduce Retro-li that shows retrieval can also help using a small-scale database, but it demands more accurate and better neighbors when searching in a smaller hence sparser non-parametric memory. This can be met by using a proper semantic similarity search. We further propose adding a regularization to the non-parametric memory for the first time: it significantly reduces perplexity when the neighbor search operations are noisy during inference, and it improves generalization when a domain shift occurs. We also show that Retro-li's non-parametric memory can potentially be implemented on analog in-memory computing hardware, exhibiting O(1) search time while causing noise in retrieving neighbors, with minimal (<1%) performance loss. Our cod

Stats

• citations1
• S2 citations—
• github stars7
• HF likes0
• heat score4.06
• arxiv keyrashiti2024retro

Related papers

• Hetarag: Hybrid Deep Retrieval-augmented Generation Across Heterogeneous Data Stores (2025)3.27
• Neurosymbolic Retrievers For Retrieval-augmented Generation (2026)0.00
• Re-ranking The Context For Multimodal Retrieval Augmented Generation (2025)0.00
• Rag-check: Evaluating Multimodal Retrieval Augmented Generation Performance (2025)0.00
• Optimizing Retrieval-augmented Generation: Analysis Of Hyperparameter Impact On Performance And Efficiency (2025)0.00
• Frustratingly Simple Retrieval Improves Challenging, Reasoning-intensive Benchmarks (2025)0.00
• Slimrag: Retrieval Without Graphs Via Entity-aware Context Selection (2025)1.91
• A Dynamic Retrieval-augmented Generation System With Selective Memory And Remembrance (2026)0.00