.png){kind=icon}
返回
Efficient Architecture Search for Continual Learning
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2022.3151511.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Continual learning with neural networks, which aims to learn a sequence of tasks, is an important learning framework in artificial intelligence (AI). However, it often confronts three challenges: 1) overcome the catastrophic forgetting problem; 2) adapt the current network to new tasks; and 3) control its model complexity. To reach these goals, we propose a novel approach named continual learning with efficient architecture search (CLEAS). CLEAS works closely with neural architecture search (NAS), which leverages reinforcement learning techniques to search for the best neural architecture that fits a new task. In particular, we design a neuron-level NAS controller that decides which old neurons from previous tasks should be reused (knowledge transfer) and which new neurons should be added (to learn new knowledge). Such a fine-grained controller allows finding a very concise architecture that can fit each new task well. Meanwhile, since we do not alter the weights of the reused neurons, we perfectly memorize the knowledge learned from the previous tasks. We evaluate CLEAS on numerous sequential classification tasks, and the results demonstrate that CLEAS outperforms other state-of-the-art alternative methods, achieving higher classification accuracy while using simpler neural architectures.
Keyword:
Task analysis
Neurons
Computer architecture
Neural networks
Deep learning
Network architecture
Knowledge engineering
Continual learning
deep learning
deep neural network
neural architecture search (NAS)
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
查看更多内容