.png){kind=icon}
返回
Dual-Conditioned Training to Exploit Pre-Trained Codebook-Based Generative Model in Image Compression
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3522238.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Learned image compression (LIC) is increasingly gaining attention. To improve the perceptual quality of reconstructions, generative LIC has been studied, using generative models such as Generative Adversarial Networks (GANs). State-of-the-art generative LIC methods have achieved remarkable performance even in low bit rate settings. Unlike most approaches trained from scratch, we propose a generative LIC that utilizes a pre-trained codebook-based generative model, Vector-Quantized GAN (VQGAN). Specifically, our model is designed to exploit its powerful image-generation capabilities to enhance compression performance. Our approach reconstructs an image from a transmitted bitstream in two steps: (1) estimating VQGAN tokens and feeding them into the pre-trained VQGAN decoder, and (2) modifying the decoder's intermediate features to address artifacts and distortions. Our preliminary experiments reveal that the information allocation between (1) and (2) is pivotal for reconstruction quality. Moreover, we found that the ideal allocation varies based on the target bit rate. Motivated by these findings, we propose a novel Dual-Conditioned training. Through the training, the model learns to adjust the total bit rate and information allocation between (1) and (2) based on two conditional inputs. Subsequently, we explore the conditional inputs to achieve the optimal results for each target bit rate. This training strategy enables us to effectively exploit the generation capability of VQGAN across different bit rates. Our method, named Dual Conditioned VQGAN-based Image Compression (DC-VIC), outperforms state-of-the-art generative LIC methods in rate-distortion-perception performance.
Keyword:
Image coding
Decoding
Image reconstruction
Training
Bit rate
Entropy
Estimation
Resource management
Distortion
Transforms
Generative adversarial networks
image compression
VQGAN
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
MoVam7, a Conserved SNARE Involved in Vacuole Assembly, Is Required for Growth, Endocytosis, ROS Accumulation, and Pathogenesis of Magnaporthe oryzae.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏Sensorless Control of Z Source Inverter fed BLDC Motor Drive by FOC - DTC Hybrid Control Strategy Using Fuzzy Logic Controller采用模糊逻辑控制器的foc-dtc混合控制策略的Z源逆变器馈电BLDC电机驱动的无传感器控制
PREAI
分享收藏Simultaneous determination of some common food dyes in commercial products by digital image analysisOAAI
OAAI分享收藏
查看更多内容