Projective Pruning for Decoupling Weights

Tommy Chu

doi:10.1007/978-3-032-06106-5_19

2026-01-01

PRE

Tommy Chu

Alexander Kovalenko *

DOI：10.1007/978-3-032-06106-5_19

原文链接

原文求助

摘要

En 中文

本文提出了一种称为投影剪枝的结构化深度神经网络稀疏化技术，该技术通过移除高度相关的权重（因其对参数子空间的贡献极小）来实现模型压缩。由于深度神经网络中存在的参数冗余和高度相关权重导致的低效问题，该方法在保持模型高性能的同时实现了参数压缩。该技术引入了一种再分配机制以保持模型性能和表达能力。在多个视觉与语言基准测试（包括大型语言模型架构）上的评估表明，与大多数其他剪枝方法不同，投影剪枝能够在确保模型性能稳定的同时实现可靠的压缩。应用该方法可提升模型的可重训练性，并达到与现有结构化剪枝方法相媲美的结果。

Keyword:

Model compression

Structured pruning

Sparsification

期刊

MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES. RESEARCH TRACK, ECML PKDD 2025, PT VI

IF:

论文数:

被引数:

机构

czech technical university prague

学者数:

6.6K

论文数: 5.3K

被引数: 3

引用论文

Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2020-06-01

OAAI

Yang He; Yuhang Ding; Ping Liu; Linchao Zhu; Hanwang Zhang; Yi Yang

Energy and Policy Considerations for Deep Learning in NLP

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

IF0

2019-01-01

OAAI

Emma Strubell; Ananya Ganesh; Andrew McCallum

MobileNetV2: Inverted Residuals and Linear Bottlenecks MobileNetV2: 反转残差和线性瓶颈

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IF0

2018-06-01

OAAI

Mark Sandler; Andrew Howard; Menglong Zhu; Andrey Zhmoginov; Liang-Chieh Chen

A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations 深度神经网络剪枝研究综述：分类、比较、分析与建议

IEEE Transactions on Pattern Analysis and Machine Intelligence

IF0

2024-12-01

OAAI

Hongrong Cheng; Miao Zhang; Javen Qinfeng Shi

Revisiting Random Channel Pruning for Neural Network Compression

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2022-06-01

OAAI

Yawei Li; Kamil Adamczewski; Wen Li; Shuhang Gu; Radu Timofte; Luc Van Gool

Deep learning 深度学习

Nature

IF0

2015-05-27

PREAI

Yann LeCun; Yoshua Bengio; Geoffrey Hinton

A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges 大型语言模型综述: 体系结构、应用、分类、开放问题和挑战

IEEE ACCESS

IF3.6

2024-01-01

117

OAAI

Raiaan, Mohaimenul Azam Khan; Mukta, Md. Saddam Hossain; Fatema, Kaniz; Fahad, Nur Mohammad; Sakib, Sadman; Mim, Most Marufatul Jannat; Ahmad, Jubaer; Ali, Mohammed Eunus; Azam, Sami

Learning Efficient Convolutional Networks through Network Slimming 通过网络瘦身学习高效卷积网络

2017 IEEE International Conference on Computer Vision (ICCV)

IF0

2017-10-01

OAAI

Zhuang Liu; Jianguo Li; Zhiqiang Shen; Gao Huang; Shoumeng Yan; Changshui Zhang

Deep Residual Learning for Image Recognition 基于深度残差学习的图像识别

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2016-06-01

OAAI

Kaiming He; Xiangyu Zhang; Shaoqing Ren; Jian Sun