Piano Transcription By Hierarchical Language Modeling With Pretrained Roll-based Encoders

Abstract

Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame-level systems require manual thresholding, while the LM-based systems struggle with long sequences. In this paper, we propose a hybrid method combining pre-trained roll-based encoders with an LM decoder to leverage the strengths of both methods. Besides, our approach employs a hierarchical prediction strategy, first predicting onset and pitch, then velocity, and finally offset. The hierarchical prediction strategy reduces computational costs by breaking down long sequences into different hierarchies. Evaluated on two benchmark roll-based encoders, our method outperforms traditional piano-roll outputs 0.01 and 0.022 in onset-offset-velocity F1 score, demonstrating its potential as a performance-enhancing plug-in for arbitrary roll-based music transcription encoder.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyli2025piano

Related papers

• Audio-to-score Alignment Of Piano Music Using Rnn-based Automatic Music Transcription (2017)0.00
• Hppnet: Modeling The Harmonic Structure And Pitch Invariance In Piano Transcription (2022)0.00
• Towards Efficient And Real-time Piano Transcription Using Neural Autoregressive Models (2024)5.84
• End-to-end Real-world Polyphonic Piano Audio-to-score Transcription With Hierarchical Decoding (2024)0.00
• Diffroll: Diffusion-based Generative Music Transcription With Unsupervised Pretraining Capability (2022)8.35
• Investigation On The Use Of Hidden-markov Models In Automatic Transcription Of Music (2017)0.00
• Deep Polyphonic ADSR Piano Note Transcription (2019)11.29
• A Holistic Approach To Polyphonic Music Transcription With Neural Networks (2019)0.00