.png){kind=icon}
Return
Projective Pruning for Decoupling Weights
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-06106-5_19.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper proposes Projective Pruning, a structured deep neural network sparsification technique that removes highly correlated weights, as they provide a minimal contribution to the parameter subspace. Due to the inefficiencies in deep neural networks caused by excessive overparametrization and highly correlated weights, the method enables parameter compression while maintaining the high performance of the models. The approach incorporates a redistribution mechanism to preserve model performance and expressiveness. Evaluations on multiple vision and language benchmarks, including large language model architectures, demonstrate that, unlike most other pruning methods, Projective Pruning delivers reliable compression while ensuring stable model performance. Applying this method improves retrainability and achieves competitive results compared to existing structured pruning methods.
Keywords:
Model compression
Structured pruning
Sparsification
Journal
M
IF:
0
Papers:
25
Citations:
0
Organization
Cited Papers
ShareSaveShareSaveA Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAIShareSaveShareSave
View more