Exploring The Viability Of Synthetic Audio Data For Audio-based Dialogue State Tracking
2023 · Jihyun Lee, Yejin Jeon, Wonjun Lee, et al.
Abstract
Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human audio datasets. We address this by investigating synthetic audio data for audio-based DST. To this end, we develop cascading and end-to-end models, train them with our synthetic audio dataset, and test them on actual human speech data. To facilitate evaluation tailored to audio modalities, we introduce a novel PhonemeF1 to capture pronunciation similarity. Experimental results showed that models trained solely on synthetic datasets can generalize their performance to human voice data. By eliminating the dependency on human speech data collection, these insights pave the way for significant practical advancements in audio-based DST. Data and code are available at https://github.com/JihyunLee1/E2E-DST.
Authors
(none)
Tags
Stats
Code
Related papers
- Comparing The Benefit Of Synthetic Training Data For Various Automatic Speech Recognition Architectures (2021)5.24
- Fake It To Make It: Using Synthetic Data To Remedy The Data Shortage In Joint Multimodal Speech-and-gesture Synthesis (2024)6.34
- On The Relevance Of Phoneme Duration Variability Of Synthesized Training Data For Automatic Speech Recognition (2023)2.26
- Speculative End-turn Detector For Efficient Speech Chatbot Assistant (2025)0.00
- Hmm-based Data Augmentation For E2E Systems For Building Conversational Speech Synthesis Systems (2022)0.00
- Enhancing Synthetic Training Data For Speech Commands: From Asr-based Filtering To Domain Adaptation In SSL Latent Space (2024)0.00
- Speechdialoguefactory: Generating High-quality Speech Dialogue Data To Accelerate Your Speech-llm Development (2025)0.00
- Sd-eval: A Benchmark Dataset For Spoken Dialogue Understanding Beyond Words (2024)11.32