Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Wen-Chin Huang

doi:10.1109/TASLP.2021.3049336

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

2021-01-01

31

OA

AI

DOI：10.1109/TASLP.2021.3049336

原文链接

查看原文

摘要

En 中文

Sequence-to-sequence (seq2seq) voice conversion (VC) models are attractive owing to their ability to convert prosody. Nonetheless, without sufficient data, seq2seq VC models can suffer from unstable training and mispronunciation problems in the converted speech, thus far from practical. To tackle these shortcomings, we propose to transfer knowledge from other speech processing tasks where large-scale corpora are easily available, typically text-to-speech (TTS) and automatic speech recognition (ASR). We argue that VC models initialized with such pretrained ASR or TTS model parameters can generate effective hidden representations for high-fidelity, highly intelligible converted speech. In this work, we examine our proposed method in a parallel, one-to-one setting. We employed recurrent neural network (RNN)-based and Transformer based models, and through systematical experiments, we demonstrate the effectiveness of the pretraining scheme and the superiority of Transformer based models over RNN-based models in terms of intelligibility, naturalness, and similarity.

Keyword:

Task analysis

Speech processing

Decoding

Training

Data models

Training data

Spectrogram

Voice conversion

sequence-to-sequence

pretraining

transformer

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

摘要

AI总结

期刊

机构