Self-supervised Learning Of Context-aware Pitch Prosody Representations

Abstract

In music and speech, meaning is derived at multiple levels of context. Affect, for example, can be inferred both by a short sound token and by sonic patterns over a longer temporal window such as an entire recording. In this letter, we focus on inferring meaning from this dichotomy of contexts. We show how contextual representations of short sung vocal lines can be implicitly learned from fundamental frequency (\(F_0\)) and thus be used as a meaningful feature space for downstream Music Information Retrieval (MIR) tasks. We propose three self-supervised deep learning paradigms which leverage pseudotask learning of these two levels of context to produce latent representation spaces. We evaluate the usefulness of these representations by embedding unseen pitch contours into each space and conducting downstream classification tasks. Our results show that contextual representation can enhance downstream classification by as much as 15% as compared to using traditional statistical contour f

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynoufi2020self

Related papers

• Traditional Machine Learning For Pitch Detection (2019)10.85
• DEEPF0: End-to-end Fundamental Frequency Estimation For Music And Speech Signals (2021)10.35
• Supervised And Unsupervised Learning Of Audio Representations For Music Understanding (2022)0.00
• Learning Music Audio Representations Via Weak Language Supervision (2021)10.07
• Adversarial Multi-task Learning For Disentangling Timbre And Pitch In Singing Voice Synthesis (2022)4.52
• Perception Of Prosodic Variation For Speech Synthesis Using An Unsupervised Discrete Representation Of F0 (2020)7.81
• Deep Audio-visual Singing Voice Transcription Based On Self-supervised Learning Models (2023)0.00
• Pretext Tasks Selection For Multitask Self-supervised Speech Representation Learning (2021)8.60