Enriching Music Descriptions With A Finetuned-llm And Metadata For Text-to-music Retrieval

Abstract

Text-to-Music Retrieval, finding music based on a given natural language query, plays a pivotal role in content discovery within extensive music databases. To address this challenge, prior research has predominantly focused on a joint embedding of music audio and text, utilizing it to retrieve music tracks that exactly match descriptive queries related to musical attributes (i.e. genre, instrument) and contextual elements (i.e. mood, theme). However, users also articulate a need to explore music that shares similarities with their favorite tracks or artists, such as \textit\{I need a similar track to Superstition by Stevie Wonder\}. To address these concerns, this paper proposes an improved Text-to-Music Retrieval model, denoted as TTMR++, which utilizes rich text descriptions generated with a finetuned large language model and metadata. To accomplish this, we obtained various types of seed text from several existing music tag and caption datasets and a knowledge graph dataset of artis

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keydoh2024enriching

Related papers

• Wikimute: A Web-sourced Dataset Of Semantic Descriptions For Music Audio (2023)5.24
• Musictm-dataset For Joint Representation Learning Among Sheet Music, Lyrics, And Musical Audio (2020)3.58
• Artistmus: A Globally Diverse, Artist-centric Benchmark For Retrieval-augmented Music Question Answering (2025)0.00
• Cross-modal Music Retrieval And Applications: An Overview Of Key Methodologies (2019)12.68
• Multimodal Metric Learning For Tag-based Music Retrieval (2020)9.76
• Expressivity-aware Music Performance Retrieval Using Mid-level Perceptual Features And Emotion Word Embeddings (2024)0.00
• Towards Robust And Truly Large-scale Audio-sheet Music Retrieval (2023)4.52
• Audio Retrieval With Natural Language Queries: A Benchmark Study (2021)16.29