.png){kind=icon}
返回
Inversion-Based Face Swapping With Diffusion Model
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3519163.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Face swapping involves replacing a face in an image with another face and ensuring the seamless integration of the source face into the target image. Previous studies have primarily utilized generative adversarial network-based models for face swapping. This paper introduces inversion-based face swapping (InFS), a novel framework employing diffusion inversion. The key contributions of our work include: 1) a facial attribute encoder that consolidates attribute information into a single embedding vector, utilizing the architecture of the pSp encoder, and 2) an enhanced face swapping pipeline that overcomes pose limitations through reenactment preprocessing addressing the challenge of incorrect face swapping at extreme angles. To preserve the target image's attribute information that may be lost during the diffusion inversion process, we incorporate the extracted information from the facial attribute encoder. This embedding vector serves as a crucial condition in the diffusion inversion process, facilitating the prediction of noisy images. Subsequently, the predicted noisy image undergoes processing using a pretrained ID conditional DDPM for face swapping. Our experimental results show that InFS outperforms state-of-the-art methods in preserving identity, expression, and shape characteristics of target images. Furthermore, the proposed InFS achieved effective face swapping results without requiring additional guidance and reduces inference time by approximately 6.96 seconds compared to previous diffusion-based approaches.
Keyword:
Faces
Diffusion models
Face recognition
Noise measurement
Facial features
Adaptation models
Pipelines
Vectors
Three-dimensional displays
Solid modeling
Face swapping
diffusion model
inversion
attribute encoder
image generation
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
分享收藏分享收藏SRDiff: Single image super-resolution with diffusion probabilistic modelsSRDiff: 具有扩散概率模型的单幅图像超分辨率.png)
NEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏Sensorless Control of Z Source Inverter fed BLDC Motor Drive by FOC - DTC Hybrid Control Strategy Using Fuzzy Logic Controller采用模糊逻辑控制器的foc-dtc混合控制策略的Z源逆变器馈电BLDC电机驱动的无传感器控制
PREAI
分享收藏
查看更多内容