Learning Deep Generative Models With Doubly Stochastic Gradient MCMC

Chao Du

doi:10.1109/TNNLS.2017.2688499

Learning Deep Generative Models With Doubly Stochastic Gradient MCMC

2017-01-01

12

PRE

AI

DOI：10.1109/TNNLS.2017.2688499

Original

Original request for help

Save

Abstract

En 中文

Deep generative models (DGMs), which are often organized in a hierarchical manner, provide a principled framework of capturing the underlying causal factors of data. Recent work on DGMs focussed on the development of efficient and scalable variational inference methods that learn a single model under some mean-field or parameterization assumptions. However, little work has been done on extending Markov chain Monte Carlo (MCMC) methods to Bayesian DGMs, which enjoy many advantages compared with variational methods. We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of DGMs in a collapsed continuous parameter space. At each MCMC sampling step, the algorithm randomly draws a mini-batch of data samples to estimate the gradient of log-posterior and further estimates the intractable expectation over hidden variables via a neural adaptive importance sampler, where the proposal distribution is parameterized by a deep neural network and learnt jointly along with the sampling process. We demonstrate the effectiveness of learning various DGMs on a wide range of tasks, including density estimation, data generation, and missing data imputation. Our method outperforms many state-of-the-art competitors.

Keywords:

Bayesian methods

deep generative models (DGMs)

deep learning

Markov chain Monte Carlo (MCMC)

stochastic gradient

Learning Deep Generative Models With Doubly Stochastic Gradient MCMC

Abstract

AI Summary

Journal

Organization