Improving Speech Emotion Recognition With Adversarial Data Augmentation Network

Yi Lu

doi:10.1109/TNNLS.2020.3027600

2022-01-01

Yi Lu

Man‐Wai Mak *

DOI：10.1109/TNNLS.2020.3027600

原文链接

查看原文

摘要

En 中文

When training data are scarce, it is challenging to train a deep neural network without causing the overfitting problem. For overcoming this challenge, this article proposes a new data augmentation network-namely adversarial data augmentation network (ADAN)- based on generative adversarial networks (GANs). The ADAN consists of a GAN, an autoencoder, and an auxiliary classifier. These networks are trained adversarially to synthesize class-dependent feature vectors in both the latent space and the original feature space, which can be augmented to the real training data for training classifiers. Instead of using the conventional cross-entropy loss for adversarial training, the Wasserstein divergence is used in an attempt to produce high-quality synthetic samples. The proposed networks were applied to speech emotion recognition using EmoDB and IEMOCAP as the evaluation data sets. It was found that by forcing the synthetic latent vectors and the real latent vectors to share a common representation, the gradient vanishing problem can be largely alleviated. Also, results show that the augmented data generated by the proposed networks are rich in emotion information. Thus, the resulting emotion classifiers are competitive with state-of-the-art speech emotion recognition systems.

Keyword:

Generators

Feature extraction

Training

Emotion recognition

Speech recognition

Generative adversarial networks

Gallium nitride

Data augmentation

generative adversarial networks (GANs)

speech emotion recognition

Wasserstein divergence