Mutilmodal Feature Extraction And Attention-based Fusion For Emotion Estimation In Videos

Abstract

The continuous improvement of human-computer interaction technology makes it possible to compute emotions. In this paper, we introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW). Sentiment analysis in human-computer interaction should, as far as possible Start with multiple dimensions, fill in the single imperfect emotion channel, and finally determine the emotion tendency by fitting multiple results. Therefore, We exploited multimodal features extracted from video of different lengths from the competition dataset, including audio, pose and images. Well-informed emotion representations drive us to propose a Attention-based multimodal framework for emotion estimation. Our system achieves the performance of 0.361 on the validation dataset. The code is available at [https://github.com/xkwangcn/ABAW-5th-RT-IAI].

Stats

• citations0
• S2 citations—
• github stars4
• HF likes0
• heat score1.40
• arxiv keyshu2023mutilmodal

Code

• xkwangcn/ABAW-5th-RT-IAI—

Related papers

• Continuous Multimodal Emotion Recognition Approach For AVEC 2017 (2017)0.00
• Audio-guided Fusion Techniques For Multimodal Emotion Analysis (2024)4.52
• SUN Team's Contribution To ABAW 2024 Competition: Audio-visual Valence-arousal Estimation And Expression Recognition (2024)0.00
• Multi-modal Continuous Valence And Arousal Prediction In The Wild Using Deep 3D Features And Sequence Modeling (2020)0.00
• A Joint Cross-attention Model For Audio-visual Fusion In Dimensional Emotion Recognition (2022)18.00
• Self-relation Attention And Temporal Awareness For Emotion Recognition Via Vocal Burst (2022)4.18
• Recursive Joint Attention For Audio-visual Fusion In Regression Based Emotion Recognition (2023)9.59
• Multimodal Fusion Method With Spatiotemporal Sequences And Relationship Learning For Valence-arousal Estimation (2024)0.00