Automatic Lyrics Transcription Using Dilated Convolutional Neural Networks With Self-attention

Abstract

Speech recognition is a well developed research field so that the current state of the art systems are being used in many applications in the software industry, yet as by today, there still does not exist such robust system for the recognition of words and sentences from singing voice. This paper proposes a complete pipeline for this task which may commonly be referred as automatic lyrics transcription (ALT). We have trained convolutional time-delay neural networks with self-attention on monophonic karaoke recordings using a sequence classification objective for building the acoustic model. The dataset used in this study, DAMP - Sing! 300x30x2 [1] is filtered to have songs with only English lyrics. Different language models are tested including MaxEnt and Recurrent Neural Networks based methods which are trained on the lyrics of pop songs in English. An in-depth analysis of the self-attention mechanism is held while tuning its context width and the number of attention heads. Using the

Stats

• citations21
• S2 citations—
• github stars0
• HF likes0
• heat score10.07
• arxiv keydemirel2020automatic

Related papers

• Deep Audio-visual Singing Voice Transcription Based On Self-supervised Learning Models (2023)0.00
• Towards Building An End-to-end Multilingual Automatic Lyrics Transcription Model (2024)0.00
• Pdaugment: Data Augmentation By Pitch And Duration Adjustments For Automatic Lyrics Transcription (2021)0.00
• State-of-the-art Speech Recognition Using Multi-stream Self-attention With Dilated 1D Convolutions (2019)11.93
• Mstre-net: Multistreaming Acoustic Modeling For Automatic Lyrics Transcription (2021)0.00
• MM-ALT: A Multimodal Automatic Lyric Transcription System (2022)8.35
• Voice And Accompaniment Separation In Music Using Self-attention Convolutional Neural Network (2020)0.00
• End-to-end Lyrics Alignment For Polyphonic Music Using An Audio-to-character Recognition Model (2019)13.11