Language Translation, And Change Of Accent For Speech-to-speech Task Using Diffusion Model

Abstract

Speech-to-speech translation (S2ST) aims to convert spoken input in one language to spoken output in another, typically focusing on either language translation or accent adaptation. However, effective cross-cultural communication requires handling both aspects simultaneously - translating content while adapting the speaker's accent to match the target language context. In this work, we propose a unified approach for simultaneous speech translation and change of accent, a task that remains underexplored in current literature. Our method reformulates the problem as a conditional generation task, where target speech is generated based on phonemes and guided by target speech features. Leveraging the power of diffusion models, known for high-fidelity generative capabilities, we adapt text-to-image diffusion strategies by conditioning on source speech transcriptions and generating Mel spectrograms representing the target speech with desired linguistic and accentual attributes. This integrate

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymishra2025language

Related papers

• Diffusion Synthesizer For Efficient Multilingual Speech To Speech Translation (2024)0.00
• Diffs2ut: A Semantic Preserving Diffusion Model For Textless Direct Speech-to-speech Translation (2023)2.26
• SLM-S2ST: A Multimodal Language Model For Direct Speech-to-speech Translation (2025)0.00
• Phonology-guided Speech-to-speech Translation For African Languages (2024)2.26
• Transpeech: Speech-to-speech Translation With Bilateral Perturbation (2022)0.00
• Daspeech: Directed Acyclic Transformer For Fast And High-quality Speech-to-speech Translation (2023)5.24
• MSLM-S2ST: A Multitask Speech Language Model For Textless Speech-to-speech Translation With Speaker Style Preservation (2024)0.00
• Streamspeech: Simultaneous Speech-to-speech Translation With Multi-task Learning (2024)7.81