.png){kind=icon}
返回
Efficient Training Acceleration via Sample-Wise Dynamic Probabilistic Pruning
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2024.3484289.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Data pruning is observed to substantially reduce the computation and memory costs of model training. Previous studies have primarily focused on constructing a series of coresets with representative samples by leveraging predefined rules for evaluating sample importance. Learning dynamics and selection bias, however, are rarely being considered. In this letter, a novel Sample-wise Dynamic Probabilistic Pruning (SwDPP) method is proposed for efficient training. Specifically, instead of hard-pruning the samples that are considered easy or well-learned, we formulate the pruning process as a probabilistic sampling problem. This is achieved by a carefully-designed soft-selection mechanism, which constantly expresses learning dynamics and relaxes selection bias. Moreover, to alleviate the accuracy drop under high pruning rates, we introduce a probabilistic Mixup strategy for information diversity maintenance. Extensive experiments conducted on CIFAR-10, CIFAR-100 and Tiny-ImageNet show that, the proposed SwDPP outperforms current state-of-the-art methods across various pruning settings. Notably, on CIFAR-10 and CIFAR-100, SwDPP achieves lossless training acceleration using only 70% of the data per epoch.
Keyword:
Training
Probabilistic logic
Data models
Computational modeling
Accuracy
Vectors
Predictive models
Optimization
Heuristic algorithms
Decision making
Data pruning
efficient training
learning dynamics
probabilistic sampling
selection bias
期刊
IF:
9.6
论文数:
1.1W
被引数:
1.7W
机构
引用论文
Change of Species and Habitat Diversity in the Pannonian Region of Eastern Lower Austria over 170 Years: Using Herbarium Records as a Witness过去170年东下奥地利潘诺尼亚地区物种与栖息地多样性的变化：以植物标本馆记录为见证
OAAI
.png){kind=icon}
OAAI分享收藏
Effect of water quality on growth of four fish species in the Iténez basin (Upper Madera, Amazon)
PREAI
分享收藏Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏分享收藏 没有更多内容
没有更多内容