A Neural Speech Codec For Noise Robust Speech Coding

Abstract

This paper considers the joint compression and enhancement problem for speech signal in the presence of noise. Recently, the SoundStream codec, which relies on end-to-end joint training of an encoder-decoder pair and a residual vector quantizer by a combination of adversarial and reconstruction losses,has shown very promising performance, especially in subjective perception quality. In this work, we provide a theoretical result to show that, to simultaneously achieve low distortion and high perception in the presence of noise, there exist an optimal two-stage optimization procedure for the joint compression and enhancement problem. This procedure firstly optimizes an encoder-decoder pair using only distortion loss and then fixes the encoder to optimize a perceptual decoder using perception loss. Based on this result, we construct a two-stage training framework for joint compression and enhancement of noisy speech signal. Unlike existing training methods which are heuristic, the propose

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2023a

Related papers

• Enhancing Into The Codec: Noise Robust Speech Coding With Vector-quantized Autoencoders (2021)10.21
• Optimizing Neural Speech Codec For Low-bitrate Compression Via Multi-scale Encoding (2024)0.00
• Neural Speech And Audio Coding: Modern AI Technology Meets Traditional Codecs (2024)7.16
• Spatialcodec: Neural Spatial Speech Coding (2023)3.69
• Speech And Noise Dual-stream Spectrogram Refine Network With Speech Distortion Loss For Robust Speech Recognition (2023)5.24
• Modeling Strategies For Speech Enhancement In The Latent Space Of A Neural Audio Codec (2025)0.00
• Apcodec+: A Spectrum-coding-based High-fidelity And High-compression-rate Neural Audio Codec With Staged Training Paradigm (2024)0.00
• Latent-domain Predictive Neural Speech Coding (2022)12.15