← all papers · overview

Cross-attention Is All You Need: Real-time Streaming Transformers For Personalised Speech Enhancement

2022 · Shucong Zhang, Malcolm Chadwick, Alberto Gil C. P. Ramos, et al.

Abstract

Personalised speech enhancement (PSE), which extracts only the speech of a target user and removes everything else from a recorded audio clip, can potentially improve users' experiences of audio AI modules deployed in the wild. To support a large variety of downstream audio tasks, such as real-time ASR and audio-call enhancement, a PSE solution should operate in a streaming mode, i.e., input audio cleaning should happen in real-time with a small latency and real-time factor. Personalisation is typically achieved by extracting a target speaker's voice profile from an enrolment audio, in the form of a static embedding vector, and then using it to condition the output of a PSE model. However, a fixed target speaker embedding may not be optimal under all conditions. In this work, we present a streaming Transformer-based PSE model and propose a novel cross-attention approach that gives adaptive target speaker representations. We present extensive experiments and show that our proposed cross

Authors

(none)

Tags

  • Speech Enhancement

Stats

  • citations0
  • S2 citations—
  • github stars0
  • HF likes0
  • heat score0.00
  • arxiv keyzhang2022cross

Related papers