A Multimodal Approach Towards Emotion Recognition Of Music Using Audio And Lyrical Content

Abstract

We propose MoodNet - A Deep Convolutional Neural Network based architecture to effectively predict the emotion associated with a piece of music given its audio and lyrical content.We evaluate different architectures consisting of varying number of two-dimensional convolutional and subsampling layers,followed by dense layers.We use Mel-Spectrograms to represent the audio content and word embeddings-specifically 100 dimensional word vectors, to represent the textual content represented by the lyrics.We feed input data from both modalities to our MoodNet architecture.The output from both the modalities are then fused as a fully connected layer and softmax classfier is used to predict the category of emotion.Using F1-score as our metric,our results show excellent performance of MoodNet over the two datasets we experimented on-The MIREX Multimodal dataset and the Million Song Dataset.Our experiments reflect the hypothesis that more complex models perform better with more training data.We al

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybhattacharya2018a

Related papers

• Music Mood Detection Based On Audio And Lyrics With Deep Neural Net (2018)0.00
• Multi-modality In Music: Predicting Emotion In Music From High-level Audio Features And Lyrics (2023)0.00
• Exploiting Synchronized Lyrics And Vocal Features For Music Emotion Detection (2019)0.00
• Multimodal Fusion With Deep Neural Networks For Audio-video Emotion Recognition (2019)0.00
• Stacked Convolutional And Recurrent Neural Networks For Music Emotion Recognition (2017)0.00
• MMVA: Multimodal Matching Based On Valence And Arousal Across Images, Music, And Musical Captions (2025)0.00
• ADFF: Attention Based Deep Feature Fusion Approach For Music Emotion Recognition (2022)0.00
• Emotech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information With Hybrid Recurrent Network (2025)8.35