.png){kind=icon}
Return
Multi-granularity knowledge distillation and prototype consistency regularization for class-incremental learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2023.05.006.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural networks (DNNs) are prone to the notorious catastrophic forgetting problem when learning new tasks incrementally. Class-incremental learning (CIL) is a promising solution to tackle the challenge and learn new classes while not forgetting old ones. Existing CIL approaches adopted stored representative exemplars or complex generative models to achieve good performance. However, storing data from previous tasks causes memory or privacy issues, and the training of generative models is unstable and inefficient. This paper proposes a method based on multi-granularity knowledge distillation and prototype consistency regularization (MDPCR) that performs well even when the previous training data is unavailable. First, we propose to design knowledge distillation losses in the deep feature space to constrain the incremental model trained on the new data. Thereby, multi -granularity is captured from three aspects: by distilling multi-scale self-attentive features, the feature similarity probability, and global features to maximize the retention of previous knowledge, effectively alleviating catastrophic forgetting. Conversely, we preserve the prototype of each old class and employ prototype consistency regularization (PCR) to ensure that the old prototypes and semantically enhanced prototypes produce consistent prediction, which excels in enhancing the robustness of old prototypes and reduces the classification bias. Extensive experiments on three CIL benchmark datasets confirm that MDPCR performs significantly better over exemplar-free methods and outperforms typical exemplar-based approaches.(c) 2023 Elsevier Ltd. All rights reserved.
Keywords:
Class -incremental learning
Knowledge distillation
Consistency regularization
Image classification
Journal
IF:
6.3
Papers:
8.2K
Citations:
3.0W
Organization
Cited Papers
ShareSaveThe effect of maternal exercise on umbilical artery blood flow in pregnancy-induced hypertension
PREAI
ShareSaveShareSavePrecipitation Versus Partitioning Kinetics during the Quenching of Low-Carbon Martensitic Steels.png)
Metals
OAAI
MetalsIF0
OAAIShareSaveContinual learning for robotics: Definition, framework, learning strategies, opportunities and challengesINFORMATION FUSIONOAAI
INFORMATION FUSIONIF15.5
OAAIShareSaveShareSave
View more