Boosting CTC-Based ASR Using LLM-Based Intermediate Loss Regularization

doi:10.1007/978-3-032-02548-7_23

2026-01-01

PRE

Duygu Altinok

DOI：10.1007/978-3-032-02548-7_23

原文链接

原文求助

摘要

En 中文

端到端（E2E）自动语音识别（ASR）系统通过将所有组件整合到单个神经网络中，彻底改变了该领域，基于注意力机制的编码器-解码器模型实现了最佳性能。然而，其自回归解码过程限制了推理速度，使其不适合实时应用。相比之下，CTC模型提供更快的非自回归解码，但在有效建模语言依赖性方面存在困难。为解决这一挑战，我们提出了一种新颖的辅助损失框架——语言感知中间损失（LAIL），利用大语言模型（LLMs）的语言知识增强CTC基ASR。通过将连接层附加到中间编码器层，LAIL将输出映射到LLM的嵌入空间，并在训练期间计算因果语言建模损失。该方法在增强语言建模的同时保留了CTC解码的计算效率。使用Conformer架构和各种LLaMA模型，我们在LibriSpeech、TEDLIUM2和WSJ语料库上显著降低了词错误率（WER），实现了CTC基ASR的最佳性能，同时计算开销极小。

Keyword:

speech recognition

CTC

Conformer

intermediate loss

non-autoregressive ASR

large language models

期刊

TEXT, SPEECH, AND DIALOGUE, TSD 2025, PT I

IF:

论文数:

被引数:

机构

暂无机构信息

引用论文

Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IF0

2018-04-01

PREAI

Linhao Dong; Shuang Xu; Bo Xu

A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

IF0

2021-12-13

OAAI

Yosuke Higuchi; Nanxin Chen; Yuya Fujita; Hirofumi Inaguma; Tatsuya Komatsu; Jaesong Lee; Jumon Nozaki; Tianzi Wang; Shinji Watanabe

Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition 高效语音识别的快速Transformer变体：具有线性可扩展注意力的Conformer

2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

IF0

2023-12-16

OAAI

Dima Rekesh; Nithin Rao Koluguri; Samuel Kriman; Somshubra Majumdar; Vahid Noroozi; He Huang; Oleksii Hrinchuk; Krishna Puvvada; Ankur Kumar; Jagadeesh Balam; Boris Ginsburg

Librispeech: An ASR corpus based on public domain audio books

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IF0

2015-04-01

PREAI

Vassil Panayotov; Guoguo Chen; Daniel Povey; Sanjeev Khudanpur

Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict

Interspeech 2020

IF0

2020-10-25

OAAI

Yosuke Higuchi; Shinji Watanabe; Nanxin Chen; Tetsuji Ogawa; Tetsunori Kobayashi

Non-Autoregressive Transformer for Speech Recognition

ieee signal processing magazine

IF3.9

2021-01-01

OAAI

Chen, Nanxin; Watanabe, Shinji; Villalba, Jesus; Zelasko, Piotr; Dehak, Najim

State-of-the-Art Speech Recognition with Sequence-to-Sequence Models

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IF0

2018-04-01

OAAI

Chung-Cheng Chiu; Tara N. Sainath; Yonghui Wu; Rohit Prabhavalkar; Patrick Nguyen; Zhifeng Chen; Anjuli Kannan; Ron J. Weiss; Kanishka Rao; Ekaterina Gonina; Navdeep Jaitly; Bo Li; Jan Chorowski; Michiel Bacchiani

Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IF0

2020-05-01

OAAI

Samuel Kriman; Stanislav Beliaev; Boris Ginsburg; Jocelyn Huang; Oleksii Kuchaiev; Vitaly Lavrukhin; Ryan Leary; Jason Li; Yang Zhang

Conformer: Convolution-augmented Transformer for Speech Recognition

Interspeech 2020

IF0

2020-10-25

OAAI

Anmol Gulati; James Qin; Chung-Cheng Chiu; Niki Parmar; Yu Zhang; Jiahui Yu; Wei Han; Shibo Wang; Zhengdong Zhang; Yonghui Wu; Ruoming Pang