Towards A Unified Conformer Structure: From ASR To ASV Task

Abstract

Transformer has achieved extraordinary performance in Natural Language Processing and Computer Vision tasks thanks to its powerful self-attention mechanism, and its variant Conformer has become a state-of-the-art architecture in the field of Automatic Speech Recognition (ASR). However, the main-stream architecture for Automatic Speaker Verification (ASV) is convolutional Neural Networks, and there is still much room for research on the Conformer based ASV. In this paper, firstly, we modify the Conformer architecture from ASR to ASV with very minor changes. Length-Scaled Attention (LSA) method and Sharpness-Aware Minimizationis (SAM) are adopted to improve model generalization. Experiments conducted on VoxCeleb and CN-Celeb show that our Conformer based ASV achieves competitive performance compared with the popular ECAPA-TDNN. Secondly, inspired by the transfer learning strategy, ASV Conformer is natural to be initialized from the pretrained ASR model. Via parameter transferring, self-a

Stats

• citations9
• S2 citations—
• github stars636
• HF likes0
• heat score13.11
• arxiv keyliao2022towards

Related papers

• Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition (2023)14.47
• Attention-based ASR With Lightweight And Dynamic Convolutions (2019)9.03
• Leveraging ASR Pretrained Conformers For Speaker Verification Through Transfer Learning And Knowledge Distillation (2023)10.74
• Conformer-based Hybrid ASR System For Switchboard Dataset (2021)9.41
• Efficient Conformer: Progressive Downsampling And Grouped Attention For Automatic Speech Recognition (2021)13.79
• Improved Conformer-based End-to-end Speech Recognition Using Neural Architecture Search (2021)0.00
• Efficient Conformer With Prob-sparse Attention Mechanism For End-to-endspeech Recognition (2021)8.09
• Nextformer: A Convnext Augmented Conformer For End-to-end Speech Recognition (2022)0.00