Revise, Reason, And Recognize: Llm-based Emotion Recognition Via Emotion-specific Prompts And ASR Error Correction
2024 · Yuanchao Li, Yuan Gong, Chao-Han Huck Yang, et al.
Abstract
Annotating and recognizing speech emotion using prompt engineering has recently emerged with the advancement of Large Language Models (LLMs), yet its efficacy and reliability remain questionable. In this paper, we conduct a systematic study on this topic, beginning with the proposal of novel prompts that incorporate emotion-specific knowledge from acoustics, linguistics, and psychology. Subsequently, we examine the effectiveness of LLM-based prompting on Automatic Speech Recognition (ASR) transcription, contrasting it with ground-truth transcription. Furthermore, we propose a Revise-Reason-Recognize prompting pipeline for robust LLM-based emotion recognition from spoken language with ASR errors. Additionally, experiments on context-aware learning, in-context learning, and instruction tuning are performed to examine the usefulness of LLM training schemes in this direction. Finally, we investigate the sensitivity of LLMs to minor prompt variations. Experimental results demonstrate the ef
Authors
(none)
Tags
Stats
Related papers
- Context And System Fusion In Post-asr Emotion Recognition With Large Language Models (2024)0.00
- Beyond Silent Letters: Amplifying Llms In Emotion Recognition With Vocal Nuances (2024)9.23
- Towards Interfacing Large Language Models With ASR Systems Using Confidence Measures And Prompting (2024)7.16
- Large Language Model Based Generative Error Correction: A Challenge And Baselines For Speech Recognition, Speaker Tagging, And Emotion Recognition (2024)7.81
- Multi-stage Large Language Model Correction For Speech Recognition (2023)0.00
- Effective Text Adaptation For Llm-based ASR Through Soft Prompt Fine-tuning (2024)5.84
- EMORL-TTS: Reinforcement Learning For Fine-grained Emotion Control In Llm-based TTS (2025)0.00
- LLM Supervised Pre-training For Multimodal Emotion Recognition In Conversations (2025)8.35