.png){kind=icon}
返回
Improving Mispronunciation Detection Using Speech Reconstruction
.png){kind=icon}
PRE
AI
DOI：10.1109/TASLP.2024.3434497.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Training related machine learning tasks simultaneously can lead to improved performance on both tasks. Text-to-speech (TTS) and mispronunciation detection and diagnosis (MDD) both operate using phonetic information and we wanted to examine whether a boost in MDD performance can be by two tasks. We propose a network that reconstructs speech from the phones produced by the MDD system and computes a speech reconstruction loss. We hypothesize that the phones produced by the MDD system will be closer to the ground truth if the reconstructed speech sounds closer to the original speech. To test this, we first extract wav2vec features from a pre-trained model and feed it to the MDD system along with the text input. The MDD system then predicts the target annotated phones and then synthesizes speech from the predicted phones. The system is therefore trained by computing both a speech reconstruction loss as well as an MDD loss. Comparing the proposed systems against an identical system but without speech reconstruction and another state-of-the-art baseline, we found that the proposed system achieves higher mispronunciation detection and diagnosis (MDD) scores. On a set of sentences unseen during training, the and speaker verification simultaneously can lead to improve proposed system achieves higher MDD scores, which suggests that reconstructing the speech signal from the predicted phones helps the system generalize to new test sentences. We also tested whether the system can generate accented speech when the input phones have mispronunciations. Results from our perceptual experiments show that speech generated from phones containing mispronunciations sounds more accented and less intelligible than phones without any mispronunciations, which suggests that the system can identify differences in phones and generate the desired speech signal.
Keyword:
Acoustics
Task analysis
Training
Speech recognition
Text to speech
Phonetics
Multitasking
Multi-task learning
mispronunciation detection
text-to-speech
speech reconstruction
期刊
I
IF:
5.1
论文数:
2.6K
被引数:
1.1W
机构
引用论文
Communications design in multi-robotic systems多机器人系统中的通信设计.png)
2020 International Conference on Communications, Signal Processing, and their Applications (ICCSPA)
2020 International Conference on Communications, Signal Processing, and their Applications (ICCSPA)IF0
PREAI
分享收藏The shadow economy in three Mediterranean countries: France, Spain and Greece. A MIMIC approach
PREAI
分享收藏A survey on state-of-the-art road surface monitoring techniques for intelligent transportation systems智能交通系统中先进道路表面监测技术综述
PREAI
分享收藏Evaluation of rice genotypes against varying population densities of root-knot nematode Meloidogyne graminicola identifies potential sources for resistance breeding对水稻基因型进行抗根结线虫 Meloidogyne graminicola 不同种群密度的评价，为抗性育种识别潜在来源
PREAI
分享收藏Personnel Real Time Tracking in Hazardous Areas Using Wearable Technologies and Machine Learning危险区域人员实时追踪：基于可穿戴技术与机器学习
PREAI
分享收藏Building a ROS-Based Testbed for Realistic Multi-Robot Simulation: Taking the Exploration as an Example构建基于ROS的真实多机器人仿真测试平台：以探索为例Robotics
OAAI
RoboticsIF0
OAAI分享收藏The impact of donor‐specific anti‐human leukocyte antigen antibodies in salvage haploidentical hematopoietic cell transplantation with posttransplant cyclophosphamide in patients with nonmalignant disordersHLA
HLAIF0
PREAI
分享收藏
查看更多内容