Diff-a-riff: Musical Accompaniment Co-creation Via Latent Diffusion Models

Abstract

Recent advancements in deep generative models present new opportunities for music production but also pose challenges, such as high computational demands and limited audio quality. Moreover, current systems frequently rely solely on text input and typically focus on producing complete musical pieces, which is incompatible with existing workflows in music production. To address these issues, we introduce "Diff-A-Riff," a Latent Diffusion Model designed to generate high-quality instrumental accompaniments adaptable to any musical context. This model offers control through either audio references, text prompts, or both, and produces 48kHz pseudo-stereo audio while significantly reducing inference time and memory usage. We demonstrate the model's capabilities through objective metrics and subjective listening tests, with extensive examples available on the accompanying website: sonycslparis.github.io/diffariff-companion/

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynistal2024diff

Related papers

• Diffrhythm: Blazingly Fast And Embarrassingly Simple End-to-end Full-length Song Generation With Latent Diffusion (2025)0.00
• Diffrhythm+: Controllable And Flexible Full-length Song Generation With Preference Optimization (2025)3.58
• Samuel: Efficient Vocal-conditioned Music Generation Via Soft Alignment Attention And Latent Diffusion (2025)0.00
• Edmsound: Spectrogram Based Diffusion Models For Efficient And High-quality Audio Synthesis (2023)0.00
• Conditional Diffusion As Latent Constraints For Controllable Symbolic Music Generation (2025)0.00
• Motionrag-diff: A Retrieval-augmented Diffusion Framework For Long-term Music-to-dance Generation (2025)0.00
• Musicldm: Enhancing Novelty In Text-to-music Generation Using Beat-synchronous Mixup Strategies (2023)13.55
• Immersediffusion: A Generative Spatial Audio Latent Diffusion Model (2024)0.00