Stream Attention-based Multi-array End-to-end Speech Recognition

Abstract

Automatic Speech Recognition (ASR) using multiple microphone arrays has achieved great success in the far-field robustness. Taking advantage of all the information that each array shares and contributes is crucial in this task. Motivated by the advances of joint Connectionist Temporal Classification (CTC)/attention mechanism in the End-to-End (E2E) ASR, a stream attention-based multi-array framework is proposed in this work. Microphone arrays, acting as information streams, are activated by separate encoders and decoded under the instruction of both CTC and attention networks. In terms of attention, a hierarchical structure is adopted. On top of the regular attention networks, stream attention is introduced to steer the decoder toward the most informative encoders. Experiments have been conducted on AMI and DIRHA multi-array corpora using the encoder-decoder architecture. Compared with the best single-array results, the proposed framework has achieved relative Word Error Rates (WERs) r

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2018stream

Related papers

• Multi-stream End-to-end Speech Recognition (2019)8.35
• Mfcca:multi-frame Cross-channel Attention For Multi-speaker ASR In Multi-party Meeting Scenario (2022)7.81
• Multi-encoder Multi-resolution Framework For End-to-end Speech Recognition (2018)0.00
• Streaming Chunk-aware Multihead Attention For Online End-to-end Speech Recognition (2020)8.60
• Automatic Channel Selection And Spatial Feature Integration For Multi-channel Speech Recognition Across Various Array Topologies (2023)8.09
• E2e-based Multi-task Learning Approach To Joint Speech And Accent Recognition (2021)0.00
• Cuside-array: A Streaming Multi-channel End-to-end Speech Recognition System With Realistic Evaluations (2024)0.00
• Integrating Source-channel And Attention-based Sequence-to-sequence Models For Speech Recognition (2019)8.09