Contrastive Audio-language Learning For Music

Abstract

As one of the most intuitive interfaces known to humans, natural language has the potential to mediate many tasks that involve human-computer interaction, especially in application-focused fields like Music Information Retrieval. In this work, we explore cross-modal learning in an attempt to bridge audio and language in the music domain. To this end, we propose MusCALL, a framework for Music Contrastive Audio-Language Learning. Our approach consists of a dual-encoder architecture that learns the alignment between pairs of music audio and descriptive sentences, producing multimodal embeddings that can be used for text-to-audio and audio-to-text retrieval out-of-the-box. Thanks to this property, MusCALL can be transferred to virtually any task that can be cast as text-based retrieval. Our experiments show that our method performs significantly better than the baselines at retrieving audio that matches a textual description and, conversely, text that matches an audio query. We also demons

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymanco2022contrastive

Related papers

• Contrastive Learning For Cross-modal Artist Retrieval (2023)0.00
• Self-supervised Contrastive Learning For Robust Audio-sheet Music Retrieval Systems (2023)5.24
• Sequential Contrastive Audio-visual Learning (2024)5.84
• Language-based Audio Retrieval With Converging Tied Layers And Contrastive Loss (2022)2.26
• Audio-visual Embedding For Cross-modal Musicvideo Retrieval Through Supervised Deep CCA (2019)11.93
• A Mathematical Perspective On Contrastive Learning (2025)0.00
• CLASP: Contrastive Language-speech Pretraining For Multilingual Multimodal Information Retrieval (2024)0.00
• Towards Robust And Truly Large-scale Audio-sheet Music Retrieval (2023)4.52