.png){kind=icon}
返回
Multi-Modal Prior-Guided Diffusion Model for Blind Image Super-Resolution
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2024.3516699.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, diffusion models have achieved remarkable success in blind image super-resolution. However, most existing methods rely solely on uni-modal degraded low-resolution images to guide diffusion models for restoring high-fidelity images, resulting in inferior realism. In this letter, we propose a Multi-modal Prior-Guided diffusion model for blind image Super-Resolution (MPGSR), which fine-tunes Stable Diffusion (SD) by utilizing the superior visual-and-textual guidance for restoring realistic high-resolution images. Specifically, our MPGSR involves two stages, i.e., multi-modal guidance extraction and adaptive guidance injection. For the former, we propose a composited transformer and further incorporate it with GPT-CLIP to extract the representative visual-and-textual guidance. For the latter, we design a feature calibration ControlNet to inject the visual guidance and employ the cross-attention layer provided by the frozen SD to inject the textual guidance, thus effectively activating the powerful text-to-image generation potential. Extensive experiments show that our MPGSR outperforms state-of-the-art methods in restoration quality and convergence time.
Keyword:
Image restoration
Feature extraction
Degradation
Transformers
Diffusion models
Visualization
Superresolution
Navigation
Image reconstruction
Adaptive systems
Blind image super-resolution
diffusion model
multi-modal guidance
transformer model
期刊
IF:
9.6
论文数:
1.1W
被引数:
1.7W
机构
引用论文
MoVam7, a Conserved SNARE Involved in Vacuole Assembly, Is Required for Growth, Endocytosis, ROS Accumulation, and Pathogenesis of Magnaporthe oryzae.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏Understanding the constitutive presentation of MHC class I immunopeptidomes in primary tissues了解MHC I类免疫蛋白在原始组织中的组成性呈现iScienceOAAI
iScienceIF0
OAAI分享收藏SRDiff: Single image super-resolution with diffusion probabilistic modelsSRDiff: 具有扩散概率模型的单幅图像超分辨率NEUROCOMPUTINGOAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏Sensorless Control of Z Source Inverter fed BLDC Motor Drive by FOC - DTC Hybrid Control Strategy Using Fuzzy Logic Controller采用模糊逻辑控制器的foc-dtc混合控制策略的Z源逆变器馈电BLDC电机驱动的无传感器控制
PREAI
分享收藏Author response: CryoEM and computer simulations reveal a novel kinase conformational switch in bacterial chemotaxis signaling作者回应：cryoEM和计算机模拟揭示了细菌趋化信号传导中的一种新型激酶构象转换
OAAI
IF0
OAAI分享收藏分享收藏
查看更多内容