A Review Of Differentiable Digital Signal Processing For Music & Speech Synthesis
2023 · Ben Hayes, Jordie Shier, György Fazekas, et al.
Abstract
The term "differentiable digital signal processing" describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music & speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably. Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.
Authors
(none)
Tags
Stats
Related papers
- Speech Synthesis And Control Using Differentiable DSP (2020)0.00
- Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-to-end Audio Style Transfer (2022)0.00
- Fast, High-quality And Parameter-efficient Articulatory Synthesis Using Differentiable DSP (2024)2.26
- Unsupervised Harmonic Parameter Estimation Using Differentiable DSP And Spectral Optimal Transport (2023)5.84
- Diffusion-based Signal Refiner For Speech Enhancement And Separation (2023)2.26
- Embedding A Differentiable Mel-cepstral Synthesis Filter To A Neural Speech Synthesis System (2022)5.24
- Differentiable Wavetable Synthesis (2021)8.82
- Audio Generation Through Score-based Generative Modeling: Design Principles And Implementation (2025)1.91