Deep Learning Enabled Semantic Communications With Speech Recognition And Synthesis

Abstract

In this paper, we develop a deep learning based semantic communication system for speech transmission, named DeepSC-ST. We take the speech recognition and speech synthesis as the transmission tasks of the communication system, respectively. First, the speech recognition-related semantic features are extracted for transmission by a joint semantic-channel encoder and the text is recovered at the receiver based on the received semantic features, which significantly reduces the required amount of data transmission without performance degradation. Then, we perform speech synthesis at the receiver, which dedicates to re-generate the speech signals by feeding the recognized text and the speaker information into a neural network module. To enable the DeepSC-ST adaptive to dynamic channel environments, we identify a robust model to cope with different channel conditions. According to the simulation results, the proposed DeepSC-ST significantly outperforms conventional communication systems and

Stats

• citations239
• S2 citations—
• github stars0
• HF likes0
• heat score17.85
• arxiv keyweng2022deep

Related papers

• Semantic Communications For Speech Signals (2020)14.35
• Semantic Communications For Speech Recognition (2021)11.93
• Robust Semantic Communications For Speech Transmission (2024)0.00
• Rate-adaptive Coding Mechanism For Semantic Communications With Multi-modal Data (2023)11.93
• A Network Of Deep Neural Networks For Distant Speech Recognition (2017)10.35
• Deep Feed-forward Sequential Memory Networks For Speech Synthesis (2018)5.84
• Large Generative Model-assisted Talking-face Semantic Communication System (2024)5.84
• STC Speaker Recognition Systems For The Voices From A Distance Challenge (2019)7.81