Matryoshka Representation Learning

Abstract

Learned representations are a central component in modern ML systems, serving a multitude of downstream tasks. When training such representations, it is often the case that computational and statistical constraints for each downstream task are unknown. In this context rigid, fixed capacity representations can be either over or under-accommodating to the task at hand. This leads us to ask: can we design a flexible representation that can adapt to multiple downstream tasks with varying computational resources? Our main contribution is Matryoshka Representation Learning (MRL) which encodes information at different granularities and allows a single embedding to adapt to the computational constraints of downstream tasks. MRL minimally modifies existing representation learning pipelines and imposes no additional cost during inference and deployment. MRL learns coarse-to-fine representations that are at least as accurate and rich as independently trained low-dimensional representations. The f

Stats

• citations7
• S2 citations—
• github stars627
• HF likes0
• heat score12.37
• arxiv keykusupati2022matryoshka

Related papers

• Beyond Matryoshka: Revisiting Sparse Coding For Adaptive Representation (2025)4.30
• Efficient Temporal-aware Matryoshka Adaptation For Temporal Information Retrieval (2026)0.00
• SMEC: Rethinking Matryoshka Representation Learning For Retrieval Embedding Compression (2025)0.00
• 2D Matryoshka Training For Information Retrieval (2024)4.06
• Matryoshka-adaptor: Unsupervised And Supervised Tuning For Smaller Embedding Dimensions (2024)2.26
• Conditional Representation Learning For Customized Tasks (2025)3.34
• Compressed Concatenation Of Small Embedding Models (2025)0.00
• TRACE: Task-adaptive Reasoning And Representation Learning For Universal Multimodal Retrieval (2026)0.00