Latent-Domain Predictive Neural Speech Coding

doi:10.1109/TASLP.2023.3277693

Latent-Domain Predictive Neural Speech Coding

2023-01-01

9

OA

AI

DOI：10.1109/TASLP.2023.3277693

Original

Original request for help

Save

Abstract

En 中文

Neural audio/speech coding has recently demonstrated its capability to deliver high quality at much lower bitrates than traditional methods. However, existing neural audio/speech codecs employ either acoustic features or learned blind features with a convolutional neural network for encoding, by which there are still temporal redundancies within encoded features. This article introduces latent-domain predictive coding into the VQ-VAE framework to fully remove such redundancies and proposes the TF-Codec for low-latency neural speech coding in an end-to-end manner. Specifically, the extracted features are encoded conditioned on a prediction from past quantized latent frames so that temporal correlations are further removed. Moreover, we introduce a learnable compression on the time-frequency input to adaptively adjust the attention paid to main frequencies and details at different bitrates. A differentiable vector quantization scheme based on distance-to-soft mapping and Gumbel-Softmax is proposed to better model the latent distributions with rate constraint. Subjective results on multilingual speech datasets show that, with low latency, the proposed TF-Codec at 1 kbps achieves significantly better quality than Opus at 9 kbps, and TF-Codec at 3 kbps outperforms both EVS at 9.6 kbps and Opus at 12 kbps. Numerous studies are conducted to demonstrate the effectiveness of these techniques.

Keywords:

Speech coding

Predictive coding

Decoding

Bit rate

Codecs

Termination of employment

Audio coding

Neural audio/speech coding

auto-encoder

predictive coding

Latent-Domain Predictive Neural Speech Coding

Abstract

Journal

Organization