Continuous Multimodal Emotion Recognition Approach For AVEC 2017

Abstract

This paper reports the analysis of audio and visual features in predicting the continuous emotion dimensions under the seventh Audio/Visual Emotion Challenge (AVEC 2017), which was done as part of a B.Tech. 2nd year internship project. For visual features we used the HOG (Histogram of Gradients) features, Fisher encodings of SIFT (Scale-Invariant Feature Transform) features based on Gaussian mixture model (GMM) and some pretrained Convolutional Neural Network layers as features; all these extracted for each video clip. For audio features we used the Bag-of-audio-words (BoAW) representation of the LLDs (low-level descriptors) generated by openXBOW provided by the organisers of the event. Then we trained fully connected neural network regression model on the dataset for all these different modalities. We applied multimodal fusion on the output models to get the Concordance correlation coefficient on Development set as well as Test set.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysingh2017continuous

Related papers

• Mutilmodal Feature Extraction And Attention-based Fusion For Emotion Estimation In Videos (2023)1.40
• A Joint Cross-attention Model For Audio-visual Fusion In Dimensional Emotion Recognition (2022)18.00
• Multimodal Fusion With Deep Neural Networks For Audio-video Emotion Recognition (2019)0.00
• Framewise Approach In Multimodal Emotion Recognition In OMG Challenge (2018)0.00
• Temporal Aggregation Of Audio-visual Modalities For Emotion Recognition (2020)8.09
• Audio-guided Fusion Techniques For Multimodal Emotion Analysis (2024)4.52
• Multi-modal Continuous Valence And Arousal Prediction In The Wild Using Deep 3D Features And Sequence Modeling (2020)0.00
• Semantic Matters: Multimodal Features For Affective Analysis (2025)0.00