.png){kind=icon}
.png){kind=icon}
Return
Bayesian compression for dynamically expandable networks
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2021.108260.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper develops Bayesian Compression for Dynamically Expandable Network (BCDEN), which can learn a compact model structure with preserving the accuracy in a continual learning scenarios. Dynamically Expandable Network (DEN) is efficiently trained by performing selective retraining, dynamically expands network capacity with only the necessary number of units, and effectively prevents semantic drift by duplicating and timestamping units in an online manner. Overcoming conventional DEN only giving point estimates, we providing the Bayesian inference under the principle framework. We validate our BCDEN on multiple public datasets under continual learning setting, on which it can outperform existing continual learning methods on a variety of tasks, and with the state-of-the-art compression results, while still maintaining comparable performance. (c) 2021 Elsevier Ltd. All rights reserved.
Keywords:
Bayesian compression
DEN
Continual learning
Selective retraining
Dynamically expands network
Semantic drift