An Empirical Study On Speech Restoration Guided By Self Supervised Speech Representation

Abstract

Enhancing speech quality is an indispensable yet difficult task as it is often complicated by a range of degradation factors. In addition to additive noise, reverberation, clipping, and speech attenuation can all adversely affect speech quality. Speech restoration aims to recover speech components from these distortions. This paper focuses on exploring the impact of self-supervised speech representation learning on the speech restoration task. Specifically, we employ speech representation in various speech restoration networks and evaluate their performance under complicated distortion scenarios. Our experiments demonstrate that the contextual information provided by the self-supervised speech representation can enhance speech restoration performance in various distortion scenarios, while also increasing robustness against the duration of speech attenuation and mismatched test conditions.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keybyun2023an

Related papers

• Selfremaster: Self-supervised Speech Restoration With Analysis-by-synthesis Approach Using Channel Modeling (2022)6.77
• Perceive And Predict: Self-supervised Speech Representation Based Loss Functions For Speech Enhancement (2023)7.16
• Efficient Personalized Speech Enhancement Through Self-supervised Learning (2021)10.21
• Automatic Data Augmentation Selection And Parametrization In Contrastive Self-supervised Speech Representation Learning (2022)5.24
• Unpaired Speech Enhancement By Acoustic And Adversarial Supervision For Speech Recognition (2018)10.21
• Personalized Speech Enhancement Through Self-supervised Data Augmentation And Purification (2021)9.92
• Voicefixer: A Unified Framework For High-fidelity Speech Restoration (2022)12.33
• Feature Normalization For Fine-tuning Self-supervised Models In Speech Enhancement (2023)2.26