Unconstrained Dysfluency Modeling For Dysfluent Speech Transcription And Detection

Abstract

Dysfluent speech modeling requires time-accurate and silence-aware transcription at both the word-level and phonetic-level. However, current research in dysfluency modeling primarily focuses on either transcription or detection, and the performance of each aspect remains limited. In this work, we present an unconstrained dysfluency modeling (UDM) approach that addresses both transcription and detection in an automatic and hierarchical manner. UDM eliminates the need for extensive manual annotation by providing a comprehensive solution. Furthermore, we introduce a simulated dysfluent dataset called VCTK++ to enhance the capabilities of UDM in phonetic transcription. Our experimental results demonstrate the effectiveness and robustness of our proposed methods in both transcription and detection tasks.

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keylian2023unconstrained

Related papers

• Stutter-solver: End-to-end Multi-lingual Dysfluency Detection (2024)5.24
• Streaming Joint Speech Recognition And Disfluency Detection (2022)0.00
• Duta-vc: A Duration-aware Typical-to-atypical Voice Conversion Approach With Diffusion Probabilistic Model (2023)0.00
• Fluentspeech: Stutter-oriented Automatic Speech Editing With Context-aware Diffusion Models (2023)12.13
• A Novel Multimodal Dynamic Fusion Network For Disfluency Detection In Spoken Utterances (2022)0.00
• End-to-end Speech Recognition And Disfluency Removal With Acoustic Language Model Pretraining (2023)0.00
• EAD-VC: Enhancing Speech Auto-disentanglement For Voice Conversion With IFUB Estimator And Joint Text-guided Consistent Learning (2024)4.52
• Improving Unsupervised Subword Modeling Via Disentangled Speech Representation Learning And Transformation (2019)5.24