.png){kind=icon}
.png){kind=icon}
Return
Force everything to one: Targeted output redirection against diffusion-based customization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2026.133393.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Diffusion-based customization can achieve highly personalized image synthesis with only a small number of user-provided samples, but it also introduces potential privacy risks. Current protection methods based on adversarial attacks primarily generate adversarial samples by maximizing the original training loss to distort the output of customized models. However, these approaches have notable limitations—they fail to adequately consider the internal features of the model, resulting in poor anti-customization performance in practice, with the generated images still retaining facial characteristics. In this paper, we propose Targeted Anti-Diffusion (TADiff), which takes a specific target image as the expected model output to avoid generating customized images. Unlike traditional non-targeted attack methods, we propose Targeted Feature Loss (TFL) in the denoising process, which trains adversarial samples by minimizing the mean square error loss between the feature maps of the protected image and the target image, effectively misguiding the model into learning incorrect features. We also design the Prompt Variation Adaptation (PVA) module to address overfitting of fixed prompts to adversarial perturbations, which enhances robustness in real scenarios. Furthermore, we apply Just Noticeable Difference (JND) to optimize the imperceptibility of perturbations while maintaining their adversarial effectiveness. Experimental results show that TADiff achieves significant improvements across two facial datasets and various text prompts, effectively protecting privacy.
Keywords:
Diffusion-based customization
Privacy protection
Adversarial attacks
Targeted feature loss
Prompt variation adaptation
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W