Diffeditor: Enhancing Speech Editing With Semantic Enrichment And Acoustic Consistency

Abstract

As text-based speech editing becomes increasingly prevalent, the demand for unrestricted free-text editing continues to grow. However, existing speech editing techniques encounter significant challenges, particularly in maintaining intelligibility and acoustic consistency when dealing with out-of-domain (OOD) text. In this paper, we introduce, DiffEditor, a novel speech editing model designed to enhance performance in OOD text scenarios through semantic enrichment and acoustic consistency. To improve the intelligibility of the edited speech, we enrich the semantic information of phoneme embeddings by integrating word embeddings extracted from a pretrained language model. Furthermore, we emphasize that interframe smoothing properties are critical for modeling acoustic consistency, and thus we propose a first-order loss function to promote smoother transitions at editing boundaries and enhance the overall fluency of the edited speech. Experimental results demonstrate that our model achie

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2024diffeditor

Related papers

• Fluenteditor: Text-based Speech Editing By Considering Acoustic And Prosody Consistency (2023)7.18
• Editspeech: A Text Based Speech Editing System Using Partial Inference And Bidirectional Fusion (2021)9.92
• Fluentspeech: Stutter-oriented Automatic Speech Editing With Context-aware Diffusion Models (2023)12.13
• Fluenteditor2: Text-based Speech Editing By Modeling Multi-scale Acoustic And Prosody Consistency (2024)3.95
• Rfm-editing: Rectified Flow Matching For Text-guided Audio Editing (2025)0.00
• Diffs2ut: A Semantic Preserving Diffusion Model For Textless Direct Speech-to-speech Translation (2023)2.26
• Usee: Unified Speech Enhancement And Editing With Conditional Diffusion Models (2023)7.50
• Improving Code-switching And Named Entity Recognition In ASR With Speech Editing Based Data Augmentation (2023)6.34