.png){kind=icon}
返回
Improving Automatic Speech Recognition Performance for Low-Resource Languages With Self-Supervised Models
.png){kind=icon}
PRE
AI
DOI：10.1109/JSTSP.2022.3184480.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Speech self-supervised learning has attracted much attention due to its promising performance in multiple downstream tasks, and has become a new growth engine for speech recognition in low-resource languages. In this paper, we exploit and analyze a series of wav2vec pre-trained models for speech recognition in 15 low-resource languages in the OpenASR21 Challenge. The investigation covers two important variables during pre-training, three fine-tuning methods, as well as applications in End-to-End and hybrid systems. First, pre-trained models with different pre-training audio data and architectures (wav2vec2.0, HuBERT and WavLM) are explored for their speech recognition performance in low-resource languages. Second, we investigate data utilization, multilingual learning, and the use of a phoneme-level recognition task in fine-tuning. Furthermore, we explore what effect fine-tuning has on the similarity of representations extracted from different transformer layers. The similarity analyses cover different pre-trained architectures and fine-tuning languages. We apply pre-trained representations to End-to-End and hybrid systems to confirm our representation analyses, which have obtained better performances as well.
Keyword:
Training
Data models
Hidden Markov models
Task analysis
Acoustics
Transformers
Computational modeling
HuBERT
low-resource language
OpenASR21
self-supervised learning
wav2vec2
0
WavLM
期刊
IF:
13.7
论文数:
1.9K
被引数:
1.1W
机构
引用论文
Wav2KWS: Transfer Learning From Speech Representations for Keyword SpottingWav2KWS: 从语音表示中进行关键词识别的迁移学习.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units休伯特: 通过隐藏单元的掩蔽预测进行自我监督的语音表示学习
OAAI
OAAI分享收藏分享收藏
查看更多内容