Zero-shot Duet Singing Voices Separation With Diffusion Models
2023 · Chin-Yun Yu, Emilian Postolache, Emanuele Rodolà, et al.
Abstract
In recent studies, diffusion models have shown promise as priors for solving audio inverse problems. These models allow us to sample from the posterior distribution of a target signal given an observed signal by manipulating the diffusion process. However, when separating audio sources of the same type, such as duet singing voices, the prior learned by the diffusion process may not be sufficient to maintain the consistency of the source identity in the separated audio. For example, the singer may change from one to another occasionally. Tackling this problem will be useful for separating sources in a choir, or a mixture of multiple instruments with similar timbre, without acquiring large amounts of paired data. In this paper, we examine this problem in the context of duet singing voices separation, and propose a method to enforce the coherency of singer identity by splitting the mixture into overlapping segments and performing posterior sampling in an auto-regressive manner, conditioni
Authors
(none)
Tags
Stats
Related papers
- Separate And Diffuse: Using A Pretrained Diffusion Model For Improving Source Separation (2023)0.00
- Hiddensinger: High-quality Singing Voice Synthesis Via Neural Audio Codec And Latent Diffusion Models (2023)0.00
- Seeing Through The Conversation: Audio-visual Speech Separation Based On Diffusion Model (2023)7.50
- Medleyvox: An Evaluation Dataset For Multiple Singing Voices Separation (2022)10.63
- A Recurrent Encoder-decoder Approach With Skip-filtering Connections For Monaural Singing Voice Separation (2017)9.41
- Monaural Singing Voice Separation With Skip-filtering Connections And Recurrent Inference Of Time-frequency Mask (2017)10.07
- Mad Twinnet: Masker-denoiser Architecture With Twin Networks For Monaural Sound Source Separation (2018)0.00
- Revisiting Representation Learning For Singing Voice Separation With Sinkhorn Distances (2020)0.00