Robust Semantic Communications For Speech Transmission

Abstract

In this paper, we propose a robust semantic communication system for speech transmission, named Ross-S2T, by delivering the essential semantic information. Specifically, we consider the speech-to-text translation (S2TT) as the transmission goal. First, a new deep semantic encoder is developed to convert speech in the source language to textual features associated with the target language, facilitating the end-to-end semantic exchange to perform the S2TT task and reducing the transmission data without performance degradation. To mitigate semantic impairments inherent in the corrupted speech, a novel generative adversarial network (GAN)-enabled deep semantic compensator is established to estimate the lost semantic information within the speech and extract deep semantic features simultaneously, which enables robust semantic transmission for corrupted speech. Furthermore, a semantic probe-aided compensator is devised to enhance the semantic fidelity of recovered semantic features and impro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyweng2024robust

Related papers

• Semantic Communications For Speech Signals (2020)14.35
• Deep Learning Enabled Semantic Communications With Speech Recognition And Synthesis (2022)17.85
• Semantic Communications For Speech Recognition (2021)11.93
• Textless Streaming Speech-to-speech Translation Using Semantic Speech Tokens (2024)3.58
• Utilizing Neural Transducers For Two-stage Text-to-speech Via Semantic Token Prediction (2024)0.00
• Socodec: A Semantic-ordered Multi-stream Speech Codec For Efficient Language Model Based Text-to-speech Synthesis (2024)6.34
• Telephonetic: Making Neural Language Models Robust To ASR And Semantic Noise (2019)0.00
• Large Generative Model-assisted Talking-face Semantic Communication System (2024)5.84