.png){kind=icon}
返回
Sparsify dynamically expandable network via variational dropout
.png){kind=icon}
PRE
AI
DOI：10.1016/j.asoc.2024.111705.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper develops a new method for lifelong learning referred to as Sparsify Dynamically Expandable Network (SDEN) via Variational Dropout, which explores a sparse model while preserving the performance. Dynamically Expandable Network (DEN) can learn a sequence of tasks via performing network retraining, network expansion by adding only the necessary neurons, and network split to effectively prevent semantic drift in an online manner. To overcome point estimation of parameters in DEN, Bayesian Compression for DEN is developed under the Bayesian framework. However, this method demands more time for model training and testing. To improve the model efficiency, we propose SDEN under the efficient sparse learning framework. We validate our SDEN in the lifelong learning scenarios with multiple frequently used benchmarks, on which it can obtain comparable classification accuracy, and less training and testing time compared with the comparison methods. Furthermore, our method can also learn a more sparse network structure which means fewer network parameters.
Keyword:
Variational dropout
Dynamically expandable network
Lifelong learning
Network retraining
Network expansion
Catastrophic forgetting or semantic drift
期刊
IF:
6.6
论文数:
1.4W
被引数:
4.8W
机构
引用论文
分享收藏Lifelong learning with Shared and Private Latent Representations learned through synaptic intelligence.png)
NEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏Continual variational dropout: a view of auxiliary local variables in continual learningMACHINE LEARNING
MACHINE LEARNINGIF2.9
PREAI
分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏Porosome: a membrane microdomain acting as the universal secretory portal in exocytosisDiscoveries
OAAI
DiscoveriesIF0
OAAI分享收藏
查看更多内容