.png){kind=icon}
Return
Boosting CTC-Based ASR Using LLM-Based Intermediate Loss Regularization
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-02548-7_23.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
End-to-end (E2E) automatic speech recognition (ASR) systems have revolutionized the field by integrating all components into a single neural network, with attention-based encoder-decoder models achieving state-of-the-art performance. However, their autoregressive decoding process limits inference speed, making them unsuitable for real-time applications. In contrast, CTCbased models offer faster, non-autoregressive decoding but struggle to model linguistic dependencies effectively. Addressing this challenge, we propose a novel auxiliary loss framework called Language-Aware Intermediate Loss (LAIL) to enhance CTC-based ASR using the linguistic knowledge of large language models (LLMs). By attaching connector layers to intermediate encoder layers, LAIL maps outputs to the embedding space of an LLM and computes a causal language modeling loss during training. This approach enhances linguistic modeling while preserving the computational efficiency of CTC decoding. Using the Conformer architecture and various LLaMA models, we demonstrate significant improvements in Word Error Rate (WER) on the LibriSpeech, TEDLIUM2, and WSJ corpora, achieving state-of-the-art performance for CTC-based ASR with minimal computational overhead.
Keywords:
speech recognition
CTC
Conformer
intermediate loss
non-autoregressive ASR
large language models
Journal
T
IF:
0
Papers:
31
Citations:
0
Organization
No organization information available
Cited Papers
ShareSaveShareSaveQuartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions.png)
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
OAAI
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)IF0
OAAIShareSave
View more