.png){kind=icon}
返回
Learning Sparse Neural Networks Using Non-Convex Regularization
.png){kind=icon}
PRE
AI
DOI：10.1109/TETCI.2021.3058672.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep Neural Networks (DNNs) is the computing paradigm that has achieved remarkable success in various fields of engineering in recent years, primarily visual recognition. DNNs owe its success to the presence of large number of weight parameters (and increased depth), which led to huge computation and memory costs for implementation. These limiting factors hinder the scalability of such algorithms on resource constraint devices (like IoT devices). In general, DNNs are believed to be over parametrized i.e., the parameters are highly redundant, thus can be structurally removed without significant loss of performance. To solve these issues, we propose to use non-convex T$\ell _{1}$ regularizer along with the additional effect of sparse group lasso to completely remove the redundant neurons/filters that is, to introduce structured sparsity. The network has been trained using the proximal gradient method, which is useful in optimizing functions with the combination of smooth and non-smooth terms. We show that proposed regularizer manages to achieve competitive performances as well as extremely compact networks. Detailed experiments are performed on several benchmark datasets that illustrate the efficiency of the approach. On the ImageNet dataset, our approach removes more than 50% of parameters of convolutional layers and 85% parameters of fully connected layers of Alexnet with no drop in accuracy.
Keyword:
Biological neural networks
Neurons
Memory management
Internet of Things
Feature extraction
Jamming
Benchmark testing
Deep neural networks
regularization
sparse group lasso
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
I
IF:
6.5
论文数:
1.4K
被引数:
4.5K
机构
引用论文
分享收藏Wider or Deeper: Revisiting the ResNet Model for Visual Recognition更广泛或更深入: 重新审视视觉识别的ResNet模型.png)
PATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAI分享收藏A novel and efficient xanthenic dye–organometallic ion‐pair complex for photoinitiating polymerization一种用于光引发聚合的新型高效的黄原胶染料-有机金属离子对配合物
PREAI
分享收藏A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine LearningIEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏
查看更多内容