.png){kind=icon}
Return
PRD3M: Prototype Representation Dataset Distillation via Diffusion Model
.png){kind=icon}
PRE
AI
DOI：10.1109/tmm.2026.3673581.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural networks (DNNs) have achieved success, but their reliance on large-scale datasets incurs substantial computational and storage overhead. Dataset distillation (DD) is an effective dataset compression technique that condenses large-scale datasets into small distilled subsets, thereby reducing computational and storage costs. While numerous matching-based DD methods have been proposed, their bi-level optimization in pixel space causes substantial computational costs, thereby limiting their applicability to large-scale datasets and complex DNNs. Recent DD methods leverage pre-trained generative models to generate distilled images, extending to large-scale datasets and improving generalization. However, these methods face the challenge of decoupling the most relevant information to the original dataset from pre-trained generative models. In this paper, we propose a prototype distillation method that utilizes representative and diverse prototypes to generate the distilled images via the stable diffusion. Specifically, we first obtain representative prototypes in the latent space to compress the original dataset by constraining distribution consistency between prototypes and the latent vectors of the original dataset. Subsequently, we ensure the diversity among the representative prototypes through a diversity loss. Finally, we leverage the obtained prototypes and text labels for stable diffusion to generate the distilled images. Compared to existing DD methods, the proposed method reduces both training time and GUP overhead while scaling effectively to large-scale datasets. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods on large-scale datasets, while improving across-architecture generalization.
Keywords:
Prototypes
Diffusion models
Training
Optimization
Noise reduction
Computational modeling
Trajectory
Redundancy
Image resolution
Noise measurement
Dataset distillation
prototype representation
latent diffusion model
Journal
IF:
9.7
Papers:
4.6K
Citations:
2.4W
Organization
Cited Papers
No cited papers available