Light-DiT: An Importance-Aware Dynamic Compression Framework for Diffusion Transformers

doi:10.1007/978-3-031-99857-7_25

2026-01-01

PRE

DOI：10.1007/978-3-031-99857-7_25

原文链接

原文求助

摘要

En 中文

扩散变换器（DiTs）在人工智能中展现出卓越的生成能力。然而，扩散过程中固有的迭代去噪过程会产生巨大的计算成本和内存开销，阻碍了快速且节能的边缘推理。为减轻去噪开销，我们提出了一种后训练框架，通过联合使用剪枝和量化进行硬件高效的DiT推理，该框架基于以下观察：在图像生成过程中，模型内的并非所有去噪块都同等重要。我们引入指标来评估DiT块和层的重要性。为实现基于重要性的动态压缩，我们将基于重要性指标的混合稀疏性剪枝和混合精度量化进行了统一。实验表明，我们的方法通过使用W3.2A4.9混合精度的剪枝实现了1.41倍的推理加速，同时仅带来极小的精度损失。此外，针对位灵活的DNN加速器的评估表明，性能可提升高达2.78倍，与未剪枝的W8A8量化相比，能耗效率可提升1.99倍。

Keyword:

Diffusion Transformers

Model Compression

Hardware-Efficient Inference

期刊

EURO-PAR 2025: PARALLEL PROCESSING, PT II

IF:

论文数:

被引数:

机构

shanghai jiao tong university

学者数:

15.7W

论文数: 11.7W

被引数: 159

chinese academy of sciences

学者数:

56.7W

论文数: 45.0W

被引数: 704

引用论文

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IF0

2024-04-14

OAAI

Hang Shao; Bei Liu; Yanmin Qian

U-Net: Convolutional Networks for Biomedical Image Segmentation U-net: 用于生物医学图像分割的卷积网络

Lecture Notes in Computer Science

IF0

2015-11-18

PREAI

Olaf Ronneberger; Philipp Fischer; Thomas Brox

DeepCache: Accelerating Diffusion Models for Free

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2024-06-16

PREAI

Xinyin Ma; Gongfan Fang; Xinchao Wang

High-Resolution Image Synthesis with Latent Diffusion Models 具有潜在扩散模型的高分辨率图像合成

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2022-06-01

OAAI

Robin Rombach; Andreas Blattmann; Dominik Lorenz; Patrick Esser; Bjorn Ommer

Bit-pragmatic deep neural network computing 位语用深度神经网络计算

Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture

IF0

2017-10-14

OAAI

Jorge Albericio; Alberto Delmás; Patrick Judd; Sayeh Sharify; Gerard O'Leary; Roman Genov; Andreas Moshovos

ImageNet Large Scale Visual Recognition Challenge ImageNet大规模视觉识别挑战

INTERNATIONAL JOURNAL OF COMPUTER VISION

IF9.3

2015-04-11

2.7W

PREAI

Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C.; Fei-Fei, Li

Scalable Diffusion Models with Transformers

2023 IEEE/CVF International Conference on Computer Vision (ICCV)

IF0

2023-10-01

OAAI

William Peebles; Saining Xie

Generative artificial intelligence: a systematic review and applications 生成式人工智能: 系统回顾与应用

Multimedia Tools and Applications

IF0

2024-08-14

OAAI

Sandeep Singh Sengar; Affan Bin Hasan; Sanjay Kumar; Fiona Carroll