Large Language Models Are Built-in Autoregressive Search Engines

Abstract

Document retrieval is a key stage of standard Web search engines. Existing dual-encoder dense retrievers obtain representations for questions and documents independently, allowing for only shallow interactions between them. To overcome this limitation, recent autoregressive search engines replace the dual-encoder architecture by directly generating identifiers for relevant documents in the candidate pool. However, the training cost of such autoregressive search engines rises sharply as the number of candidate documents increases. In this paper, we find that large language models (LLMs) can follow human instructions to directly generate URLs for document retrieval. Surprisingly, when providing a few \{Query-URL\} pairs as in-context demonstrations, LLMs can generate Web URLs where nearly 90% of the corresponding documents contain correct answers to open-domain questions. In this way, LLMs can be thought of as built-in search engines, since they have not been explicitly trained to map

Stats

• citations23
• S2 citations—
• github stars36
• HF likes0
• heat score13.49
• arxiv keyziems2023large

Related papers

• Lightretriever: A Llm-based Text Retrieval Architecture With Extremely Faster Query Inference (2025)0.00
• Improving Tool Retrieval By Leveraging Large Language Models For Query Generation (2024)0.00
• Promptreps: Prompting Large Language Models To Generate Dense And Sparse Representations For Zero-shot Document Retrieval (2024)10.61
• Scalingnote: Scaling Up Retrievers With Large Language Models For Real-world Dense Retrieval (2024)0.00
• Large Reasoning Embedding Models: Towards Next-generation Dense Retrieval Paradigm (2025)0.00
• Making Large Language Models Efficient Dense Retrievers (2025)0.00
• Transforming Llms Into Cross-modal And Cross-lingual Retrieval Systems (2024)4.52
• Llm-augmented Retrieval: Enhancing Retrieval Models Through Language Models And Doc-level Embedding (2024)0.00