The Potential Of Neural Speech Synthesis-based Data Augmentation For Personalized Speech Enhancement

Abstract

With the advances in deep learning, speech enhancement systems benefited from large neural network architectures and achieved state-of-the-art quality. However, speaker-agnostic methods are not always desirable, both in terms of quality and their complexity, when they are to be used in a resource-constrained environment. One promising way is personalized speech enhancement (PSE), which is a smaller and easier speech enhancement problem for small models to solve, because it focuses on a particular test-time user. To achieve the personalization goal, while dealing with the typical lack of personal data, we investigate the effect of data augmentation based on neural speech synthesis (NSS). In the proposed method, we show that the quality of the NSS system's synthetic data matters, and if they are good enough the augmented dataset can be used to improve the PSE system that outperforms the speaker-agnostic baseline. The proposed PSE systems show significant complexity reduction while preser

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keykuznetsova2022the

Related papers

• Generative Data Augmentation Challenge: Zero-shot Speech Synthesis For Personalized Speech Enhancement (2025)4.52
• Personalized Speech Enhancement Without A Separate Speaker Embedding Model (2024)5.24
• Personalized Speech Enhancement Through Self-supervised Data Augmentation And Purification (2021)9.92
• Plugin Speech Enhancement: A Universal Speech Enhancement Framework Inspired By Dynamic Neural Network (2024)0.00
• Text Is All You Need: Personalizing ASR Models Using Controllable Speech Synthesis (2023)7.16
• Speaker Verification-derived Loss And Data Augmentation For Dnn-based Multispeaker Speech Synthesis (2021)3.58
• Sef-pnet: Speaker Encoder-free Personalized Speech Enhancement With Local And Global Contexts Aggregation (2025)2.26
• Dynamic Acoustic Compensation And Adaptive Focal Training For Personalized Speech Enhancement (2022)4.52