Fusing Memory And Attention: A Study On LSTM, Transformer And Hybrid Architectures For Symbolic Music Generation

Abstract

Machine learning techniques, such as Transformers and Long Short-Term Memory (LSTM) networks, play a crucial role in Symbolic Music Generation (SMG). Existing literature indicates a difference between LSTMs and Transformers regarding their ability to model local melodic continuity versus maintaining global structural coherence. However, their specific properties within the context of SMG have not been systematically studied. This paper addresses this gap by providing a fine-grained comparative analysis of LSTMs versus Transformers for SMG, examining local and global properties in detail using 17 musical quality metrics on the Deutschl dataset. We find that LSTM networks excel at capturing local patterns but fail to preserve long-range dependencies, while Transformers model global structure effectively but tend to produce irregular phrasing. Based on this analysis and leveraging their respective strengths, we propose a Hybrid architecture combining a Transformer Encoder with an LSTM Dec

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyghoshal2026fusing

Related papers

• The Effect Of Explicit Structure Encoding Of Deep Neural Networks For Symbolic Music Generation (2018)11.49
• Nested Music Transformer: Sequentially Decoding Compound Tokens In Symbolic Music And Audio Generation (2024)0.00
• Amadeus: Autoregressive Model With Bidirectional Attribute Modelling For Symbolic Music (2025)0.00
• CSL-L2M: Controllable Song-level Lyric-to-melody Generation Based On Conditional Transformer With Fine-grained Lyric And Musical Controls (2024)2.26
• Music Generation Based On Generative Adversarial Networks With Transformer (2023)0.00
• Auto-regressive Vs Flow-matching: A Comparative Study Of Modeling Paradigms For Text-to-music Generation (2025)0.00
• Inspiremusic: Integrating Super Resolution And Large Language Model For High-fidelity Long-form Music Generation (2025)6.26
• Musetok: Symbolic Music Tokenization For Generation And Semantic Understanding (2025)0.00