Speaker Identity Preservation In Dysarthric Speech Reconstruction By Adversarial Speaker Adaptation
2022 · Disong Wang, Songxiang Liu, Xixin Wu, et al.
Abstract
Dysarthric speech reconstruction (DSR), which aims to improve the quality of dysarthric speech, remains a challenge, not only because we need to restore the speech to be normal, but also must preserve the speaker's identity. The speaker representation extracted by the speaker encoder (SE) optimized for speaker verification has been explored to control the speaker identity. However, the SE may not be able to fully capture the characteristics of dysarthric speakers that are previously unseen. To address this research problem, we propose a novel multi-task learning strategy, i.e., adversarial speaker adaptation (ASA). The primary task of ASA fine-tunes the SE with the speech of the target dysarthric speaker to effectively capture identity-related information, and the secondary task applies adversarial training to avoid the incorporation of abnormal speaking patterns into the reconstructed speech, by regularizing the distribution of reconstructed speech to be close to that of reference spe
Authors
(none)
Tags
Stats
Related papers
- Adversarial Speaker Adaptation (2019)10.21
- Enhancing Dysarthric Speech Recognition For Unseen Speakers Via Prototype-based Adaptation (2024)9.45
- Speaker Adaptation Using Spectro-temporal Deep Features For Dysarthric And Elderly Speech Recognition (2022)12.02
- Learning Explicit Prosody Models And Deep Speaker Embeddings For Atypical Voice Conversion (2020)7.16
- SEEF-ALDR: A Speaker Embedding Enhancement Framework Via Adversarial Learning Based Disentangled Representation (2019)3.58
- Colm-dsr: Leveraging Neural Codec Language Modeling For Multi-modal Dysarthric Speech Reconstruction (2024)3.58
- ASRRL-TTS: Agile Speaker Representation Reinforcement Learning For Text-to-speech Speaker Adaptation (2024)0.00
- UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization (2024)0.00