Efficient CTC Regularization Via Coarse Labels For End-to-end Speech Translation

Abstract

For end-to-end speech translation, regularizing the encoder with the Connectionist Temporal Classification (CTC) objective using the source transcript or target translation as labels can greatly improve quality metrics. However, CTC demands an extra prediction layer over the vocabulary space, bringing in nonnegligible model parameters and computational overheads, although this layer is typically not used for inference. In this paper, we re-examine the need for genuine vocabulary labels for CTC for regularization and explore strategies to reduce the CTC label space, targeting improved efficiency without quality degradation. We propose coarse labeling for CTC (CoLaCTC), which merges vocabulary labels via simple heuristic rules, such as using truncation, division or modulo (MOD) operations. Despite its simplicity, our experiments on 4 source and 8 target languages show that CoLaCTC with MOD particularly can compress the label space aggressively to 256 and even further, gaining training ef

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2023efficient

Related papers

• CR-CTC: Consistency Regularization On CTC For Improved Speech Recognition (2024)6.30
• CTC-GMM: CTC Guided Modality Matching For Fast And Accurate Streaming Speech Translation (2024)3.58
• CTC Blank Triggered Dynamic Layer-skipping For Efficient Ctc-based Speech Recognition (2024)0.00
• Gram-ctc: Automatic Unit Selection And Target Decomposition For Sequence Labelling (2017)0.00
• Blank Collapse: Compressing CTC Emission For The Faster Decoding (2022)0.00
• Investigating The Reordering Capability In Ctc-based Non-autoregressive End-to-end Speech Translation (2021)0.00
• Label-context-dependent Internal Language Model Estimation For CTC (2025)0.00
• BERT Meets CTC: New Formulation Of End-to-end Speech Recognition With Pre-trained Masked Language Model (2022)0.00