Magic-mm-embedding: Towards Visual-token-efficient Universal Multimodal Embedding With Mllms

Abstract

Multimodal Large Language Models (MLLMs) have shown immense promise in universal multimodal retrieval, which aims to find relevant items of various modalities for a given query. But their practical application is often hindered by the substantial computational cost incurred from processing a large number of tokens from visual inputs. In this paper, we propose Magic-MM-Embedding, a series of novel models that achieve both high efficiency and state-of-the-art performance in universal multimodal embedding. Our approach is built on two synergistic pillars: (1) a highly efficient MLLM architecture incorporating visual token compression to drastically reduce inference latency and memory footprint, and (2) a multi-stage progressive training strategy designed to not only recover but significantly boost performance. This coarse-to-fine training paradigm begins with extensive continue pretraining to restore multimodal understanding and generation capabilities, progresses to large-scale contrasti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2026magic

Related papers

• Breaking The Modality Barrier: Universal Embedding Learning With Multimodal Llms (2025)4.52
• U-MARVEL: Unveiling Key Factors For Universal Multimodal Retrieval Via Embedding Learning With Mllms (2025)3.11
• Indexing Multimodal Language Models For Large-scale Image Retrieval (2026)0.00
• Mm-embed: Universal Multimodal Retrieval With Multimodal Llms (2024)0.00
• CREM: Compression-driven Representation Enhancement For Multimodal Retrieval And Comprehension (2026)0.00
• Compressing Then Matching: An Efficient Pre-training Paradigm For Multimodal Embedding (2025)0.00
• Vlm2vec: Training Vision-language Models For Massive Multimodal Embedding Tasks (2024)0.00
• RETLLM: Training And Data-free Mllms For Multimodal Information Retrieval (2026)1.57