.png){kind=icon}
Return
PiT: Progressive diffusion transformer
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2026.114732.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
•
DiT does not rely on long-distance attention, Static Window Attention is proposed.
•
Pseudo Shifted Window Attention is introduced to enhance high-frequency information.
•
We pioneer the concept of Kth-order attention in visual generation.
•
Progressive Coverage Channel Allocation enables effective Kth-order attention.
•
PiT, a new vision backbone, achieves SOTA performance and is 2.46× faster than DiT.
Abstract
Diffusion Transformers (DiTs) achieve remarkable performance within image generation. Conventionally, DiTs are constructed by stacking serial isotropic transformers, which face significant quadratic computational cost. However, through empirical analysis, we find that DiTs do not rely as heavily on long-distance information as previously believed. In fact, most layers exhibit significant redundancy in long-distance computation. Additionally, conventional attention mechanisms suffer from low-frequency inertia, limiting their efficiency. To address these issues, we propose Pseudo Shifted Window Attention (PSWA), which fundamentally mitigates long-distance attention redundancy. PSWA achieves moderate global–local information through Static Window Attention. It further utilizes a high-frequency bridging branch to enrich the high-frequency information and strengthen inter-window connections. Furthermore, we pioneer the concept of Kth-order Attention and propose the Progressive Coverage Channel Allocation (PCCA) strategy that captures high-order attention by reallocating the existing channel budget. Based on these innovations, we propose a series of Pseudo Progressive Diffusion Transformer (PiT). Extensive experiments show superior performance of PiT; for example, PiT-L achieves 54% FID improvement over DiT-XL/2 with less computation.
Keywords:
Image generation
Diffusion transformer
Attention similarity
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSave
View more