.png){kind=icon}
.png){kind=icon}
Return
Key-Value Mapping-Based Text-to-Image Diffusion Model Backdoor Attacks
.png){kind=icon}
OA
AI
DOI：10.3390/a19010074.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Text-to-image (T2I) generation, a core component of generative artificial intelligence(AI), is increasingly important for creative industries and human-computer interaction. Despite impressive progress in realism and diversity, diffusion models still exhibit critical security blind spots particularly in the Transformer key-value mapping mechanism that underpins cross-modal alignment. Existing backdoor attacks often rely on large-scale data poisoning or extensive fine-tuning, leading to low efficiency and limited stealth. To address these challenges, we propose two efficient backdoor attack methods AttnBackdoor and SemBackdoor grounded in the Transformer's key-value storage principle. AttnBackdoor injects precise mappings between trigger prompts and target instances by fine-tuning the key-value projection matrices in U-Net cross-attention layers (approximate to 5% of parameters). SemBackdoor establishes semantic-level mappings by editing the text encoder's MLP projection matrix (approximate to 0.3% of parameters). Both approaches achieve high attack success rates (>90%), with SemBackdoor reaching 98.6% and AttnBackdoor 97.2%. They also reduce parameter updates and training time by 1-2 orders of magnitude compared to prior work while preserving benign generation quality. Our findings reveal dual vulnerabilities at visual and semantic levels and provide a foundation for developing next generation defenses for secure generative AI.
Keywords:
backdoor attack
diffusion models
text-to-image generation
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.