.png){kind=icon}
.png){kind=icon}
Return
FRMDN: Flow-based Recurrent Mixture Density Network
.png){kind=icon}
OA
AI
DOI：10.1016/j.eswa.2023.121360.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The class of recurrent mixture density networks is an important class of probabilistic models used extensively in sequence modeling and sequence-to-sequence mapping applications. In this class of models, the density of a target sequence in each time-step is modeled by a Gaussian mixture model with the parameters given by a recurrent neural network. In this paper, we generalize recurrent mixture density networks by using a normalizing flow to non-linearly transform the target space. Furthermore to improve the modeling power, we adopting a suitable covariance matrix decomposition involving a summation of a low-rank and a diagonal matrix. Using these two techniques, we still have a tractable log-likelihood. We also applied the proposed model on some speech and image data, and observed that the model has significant modeling power outperforming other state-of-the-art methods in terms of the log-likelihood on some data. The log-likelihood improvement over other methods is 3523 units for TIMIT speech dataset, and is 1118 and 176 units for MNIST and CIFAR10 image datasets. We were only underperformed on one of the speech datasets by 1209 units.
Keywords:
Recurrent mixture density network
Normalizing flow
Density estimation
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W