.png){kind=icon}
Return
Reparameterizing and dynamically quantizing image features for image generation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109962.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
For autoregressive image generation, vector-quantized VAEs (VQ-VAEs) quantize image features with discrete codebook entries and reconstruct images from quantized features. However, they treat each codebook entry separately, which causes losses of image details. In this paper, we propose to reparameterize image features with weight vectors to treat all codebook entries as an entity, and present a novel dynamically vector quantized VAE (DVQ-VAE) to quantize reparameterized image features. Specifically, each image feature corresponds to a weight vector and we sum weighted codebook entries to obtain values of image features. In this way, image features can incorporate information from different codebook entries. Additionally, a novel continuous weight regularization loss is proposed to improve the reconstruction of image details. Our method achieves competitive results with prior state-of-the-art works for image generation and extensive experiments are conducted to take a deep insight into our DVQ-VAE.
Keywords:
Vector quantization
Variational auto-encoder
Unconditional image generation
Text-to-image generation
Autoregressive generation
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
Cited Papers
ShareSave
Effects of heated seat and foot heater on thermal comfort and heater energy consumption in vehicle.png)
Ergonomics
ErgonomicsIF0
PREAI
ShareSaveGL-GAN: Adaptive global and local bilevel optimization for generative adversarial networkPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSave
ShareSaveSemi-supervised cross-modal image generation with generative adversarial networksPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSave no more
no more