Learning deep generative models based on binomial log-likelihood

Hwichang Jeong

doi:10.1016/j.neucom.2025.131009

Learning deep generative models based on binomial log-likelihood

2025-07-15

0

PRE

AI

DOI：10.1016/j.neucom.2025.131009

Original

Original request for help

Save

Abstract

En 中文

Likelihood-based learning algorithms for deep generative models mostly use the Gaussian log-likelihood. One notable exception is the binomial log-likelihood used in the Wasserstein autoencoder; however, it is not commonly used in practice because it does not generalize well. In this paper, we reconsider the binomial log-likelihood for learning deep generative models and study its theoretical properties. We propose two modifications to the original binomial log-likelihood and derive the convergence rates of the corresponding maximum likelihood estimators. These theoretical results explain why the original binomial log-likelihood performs poorly. In addition, motivated by the modified binomial log-likelihood, we propose a parametric heterogeneous Gaussian log-likelihood, which is novel in learning deep generative models. By analyzing various benchmark image datasets, we show that the proposed parametric heterogeneous Gaussian log-likelihood outperforms the standard homogeneous Gaussian log-likelihood. Additionally, we provide several pieces of evidence to explain why the proposed heterogeneous Gaussian log-likelihood works better than others.

Keywords:

binomial log-likelihood

deep generative models

maximum likelihood estimation

convergence rates

heterogeneous Gaussian log-likelihood

Learning deep generative models based on binomial log-likelihood

Abstract

Journal

Organization

Cited Papers