Multi-channel Opus Compression For Far-field Automatic Speech Recognition With A Fixed Bitrate Budget

Abstract

Automatic speech recognition (ASR) in the cloud allows the use of larger models and more powerful multi-channel signal processing front-ends compared to on-device processing. However, it also adds an inherent latency due to the transmission of the audio signal, especially when transmitting multiple channels of a microphone array. One way to reduce the network bandwidth requirements is client-side compression with a lossy codec such as Opus. However, this compression can have a detrimental effect especially on multi-channel ASR front-ends, due to the distortion and loss of spatial information introduced by the codec. In this publication, we propose an improved approach for the compression of microphone array signals based on Opus, using a modified joint channel coding approach and additionally introducing a multi-channel spatial decorrelating transform to reduce redundancy in the transmission. We illustrate the effect of the proposed approach on the spatial information retained in multi

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keydrude2021multi

Related papers

• Frequency Domain Multi-channel Acoustic Modeling For Distant Speech Recognition (2019)9.92
• Automatic Channel Selection And Spatial Feature Integration For Multi-channel Speech Recognition Across Various Array Topologies (2023)8.09
• Multi-geometry Spatial Acoustic Modeling For Distant Speech Recognition (2019)6.34
• Spatialcodec: Neural Spatial Speech Coding (2023)3.69
• End-to-end Dereverberation, Beamforming, And Speech Recognition With Improved Numerical Stability And Advanced Frontend (2021)10.97
• Task And Perception-aware Distributed Source Coding For Correlated Speech Under Bandwidth-constrained Channels (2025)0.00
• Msr-codec: A Low-bitrate Multi-stream Residual Codec For High-fidelity Speech Generation With Information Disentanglement (2025)2.35
• Leveraging Redundancy In Multiple Audio Signals For Far-field Speech Recognition (2023)0.00