PLUM: Adapting Pre-trained Language Models For Industrial-scale Generative Recommendations

Abstract

Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyhe2025plum

Related papers

• Bridging Language And Items For Retrieval And Recommendation: Benchmarking Llms As Semantic Encoders (2024)0.00
• STAR: A Simple Training-free Approach For Recommendations Using Large Language Models (2024)0.00
• Talkplay-tools: Conversational Music Recommendation With LLM Tool Calling (2025)0.00
• Notellm: A Retrievable Large Language Model For Note Recommendation (2024)9.41
• Lamra: Large Multimodal Model As Your Advanced Retrieval Assistant (2024)7.50
• LMAR: Language Model Augmented Retriever For Domain-specific Knowledge Indexing (2025)1.57
• Vlm4rec: Multimodal Semantic Representation For Recommendation With Large Vision-language Models (2026)1.82
• CSPLADE: Learned Sparse Retrieval With Causal Language Models (2025)0.00