.png){kind=icon}
Return
Distilled Meta-learning for Multi-Class Incremental Learning
.png){kind=icon}
PRE
AI
DOI：10.1145/3576045.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Meta-learning approaches have recently achieved promising performance inmulti-class incremental learning. However, meta-learners still suffer from catastrophic forgetting, i.e., they tend to forget the learned knowledge from the old tasks when they focus on rapidly adapting to the new classes of the current task. To solve this problem, we propose a novel distilled meta-learning (DML) framework for multi-class incremental learning that integrates seamlessly meta-learning with knowledge distillation in each incremental stage. Specifically, during inner-loop training, knowledge distillation is incorporated into the DML to overcome catastrophic forgetting. During outer-loop training, a meta-update rule is designed for the meta-learner to learn across tasks and quickly adapt to new tasks. By virtue of the bilevel optimization, our model is encouraged to reach a balance between the retention of old knowledge and the learning of new knowledge. Experimental results on four benchmark datasets demonstrate the effectiveness of our proposal and show that our method significantly outperforms other state-of-the-art incremental learning methods.
Keywords:
Incremental learning
meta-learning
knowledge distillation
catastrophic forgetting
stability-plasticity dilemma
Journal
IF:
6
Papers:
2.0K
Citations:
5.4K
Organization
Cited Papers
ShareSaveShareSaveThe Alkaloid Compound Harmane Increases the Lifespan of Caenorhabditis elegans during Bacterial Infection, by Modulating the Nematode’s Innate Immune Response.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAIShareSaveShareSaveShareSaveShareSave
View more