.png){kind=icon}
返回
Multi-granularity knowledge distillation and prototype consistency regularization for class-incremental learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2023.05.006.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep neural networks (DNNs) are prone to the notorious catastrophic forgetting problem when learning new tasks incrementally. Class-incremental learning (CIL) is a promising solution to tackle the challenge and learn new classes while not forgetting old ones. Existing CIL approaches adopted stored representative exemplars or complex generative models to achieve good performance. However, storing data from previous tasks causes memory or privacy issues, and the training of generative models is unstable and inefficient. This paper proposes a method based on multi-granularity knowledge distillation and prototype consistency regularization (MDPCR) that performs well even when the previous training data is unavailable. First, we propose to design knowledge distillation losses in the deep feature space to constrain the incremental model trained on the new data. Thereby, multi -granularity is captured from three aspects: by distilling multi-scale self-attentive features, the feature similarity probability, and global features to maximize the retention of previous knowledge, effectively alleviating catastrophic forgetting. Conversely, we preserve the prototype of each old class and employ prototype consistency regularization (PCR) to ensure that the old prototypes and semantically enhanced prototypes produce consistent prediction, which excels in enhancing the robustness of old prototypes and reduces the classification bias. Extensive experiments on three CIL benchmark datasets confirm that MDPCR performs significantly better over exemplar-free methods and outperforms typical exemplar-based approaches.(c) 2023 Elsevier Ltd. All rights reserved.
Keyword:
Class -incremental learning
Knowledge distillation
Consistency regularization
Image classification
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
The effect of maternal exercise on umbilical artery blood flow in pregnancy-induced hypertension
PREAI
分享收藏Precipitation Versus Partitioning Kinetics during the Quenching of Low-Carbon Martensitic Steels.png)
Metals
OAAI
MetalsIF0
OAAI分享收藏Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges机器人持续学习: 定义、框架、学习策略、机遇与挑战INFORMATION FUSIONOAAI
INFORMATION FUSIONIF15.5
OAAI分享收藏
查看更多内容