.png){kind=icon}
返回
Bayesian mixture variational autoencoders for multi-modal learning
.png){kind=icon}
OA
AI
DOI：10.1007/s10994-022-06272-y.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper provides an in-depth analysis on how to effectively acquire and generalize cross-modal knowledge for multi-modal learning. Mixture-of-Expert (MoE) and Product-of-Expert (PoE) are two popular directions in generalizing multi-modal information. Existing works based on MoE or PoE have shown notable improvement on data generation, while new challenges such as high training cost, overconfident experts, and encoding modal-specific features also emerge. In this work, we propose Bayesian mixture variational autoencoder (BMVAE) which learns to select or combine experts via Bayesian inference. We show that the proposed idea can naturally encourage models to learn modal-specific knowledge and avoid overconfident experts. Also, we show that the idea is compatible with both MoE and PoE frameworks. When being a MoE model, BMVAE can be optimized by a tight lower bound and is efficient to train. The PoE BMVAE has the same advantages and a theoretical connection to existing works. In the experiments, we show that BMVAE achieves state-of-the-art performance.
Keyword:
Multi-modal learning
Mixture-of-Gaussian
Product-of-Gaussian
Deep generative model
Disentangled representation
期刊
IF:
2.9
论文数:
2.7K
被引数:
3.4W
机构
引用论文
.png){kind=icon}
Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏Inosine pranobex enhances human NK cell cytotoxicity by inducing metabolic activation and NKG2D ligand expression肌苷pranobex通过诱导代谢活化和NKG2D配体表达增强人NK细胞的细胞毒性
OAAI
OAAI分享收藏 没有更多内容
没有更多内容