Multimodal Metric Learning For Tag-based Music Retrieval

Abstract

Tag-based music retrieval is crucial to browse large-scale music libraries efficiently. Hence, automatic music tagging has been actively explored, mostly as a classification task, which has an inherent limitation: a fixed vocabulary. On the other hand, metric learning enables flexible vocabularies by using pretrained word embeddings as side information. Also, metric learning has already proven its suitability for cross-modal retrieval tasks in other domains (e.g., text-to-image) by jointly learning a multimodal embedding space. In this paper, we investigate three ideas to successfully introduce multimodal metric learning for tag-based music retrieval: elaborate triplet sampling, acoustic and cultural music information, and domain-specific word embeddings. Our experimental results show that the proposed ideas enhance the retrieval system quantitatively, and qualitatively. Furthermore, we release the MSD500, a subset of the Million Song Dataset (MSD) containing 500 cleaned tags, 7 manual

Stats

• citations19
• S2 citations—
• github stars0
• HF likes0
• heat score9.76
• arxiv keywon2020multimodal

Related papers

• Disentangled Multidimensional Metric Learning For Music Similarity (2020)10.61
• Self-supervised Auxiliary Loss For Metric Learning In Music Similarity-based Retrieval And Auto-tagging (2023)0.00
• Musictm-dataset For Joint Representation Learning Among Sheet Music, Lyrics, And Musical Audio (2020)3.58
• Metric Learning Vs Classification For Disentangled Music Representation Learning (2020)0.00
• Towards Robust And Truly Large-scale Audio-sheet Music Retrieval (2023)4.52
• Anchor-aware Deep Metric Learning For Audio-visual Retrieval (2024)6.34
• Exploring Modality-agnostic Representations For Music Classification (2021)0.00
• Contrastive Learning For Cross-modal Artist Retrieval (2023)0.00