Planning Ahead In Generative Retrieval: Guiding Autoregressive Generation Through Simultaneous Decoding

Abstract

This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding. To this aim, PAG constructs a set-based and sequential identifier for each document. Motivated by the bag-of-words assumption in information retrieval, the set-based identifier is built on lexical tokens. The sequential identifier, on the other hand, is obtained via quantizing relevance-based representations of documents. Extensive experiments on MSMARCO and TREC Deep Learning Track data reveal that PAG outperforms the state-of-the-art generative retrieval model by a large margin (e.g., 15.6% MRR improvements on MS MARCO), while achieving 22x speed up in terms of query latency.

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keyzeng2024planning

Related papers

• Nonparametric Decoding For Generative Retrieval (2022)5.84
• Generative Retrieval Meets Multi-graded Relevance (2024)2.26
• Generative Retrieval As Dense Retrieval (2023)0.00
• ASI++: Towards Distributionally Balanced End-to-end Generative Retrieval (2024)0.00
• AR-RAG: Autoregressive Retrieval Augmentation For Image Generation (2025)0.00
• Learning To Tokenize For Generative Retrieval (2023)4.52
• Generative Retrieval As Multi-vector Dense Retrieval (2024)8.60
• Listwise Generative Retrieval Models Via A Sequential Learning Process (2024)8.60