.png){kind=icon}
Return
Potential Knowledge Extraction Network for Class-Incremental Learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.128923.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Class-Incremental Learning (CIL) aims to dynamically learn new classes without forgetting the old ones, and it is typically achieved by extracting knowledge from old data and continuously transferring it to new tasks. In the replay-based approaches, selecting appropriate exemplars is of great importance since exemplars represent the most direct form of retaining old knowledge. In this paper, we propose a novel CIL framework: Potential Knowledge Extraction Network (PKENet), which addresses the issue of neglecting the knowledge of inter-sample relation inmost existing works and suggests an innovative approach for exemplar selection. Specifically, to address the challenge of knowledge transfer, we design a relation consistency loss and a hybrid cross-entropy loss, where the former works by extracting structural knowledge from the old model while the latter captures graph-wise knowledge, enabling the new model to acquire more old knowledge. To enhance the anti-forgetting effect of exemplar set, we devise a maximum-forgetting-priority method for selecting samples most susceptible to interference from the model's update. To overcome the prediction bias problem in CIL, we introduce the Total Direct Effect inference method into our model. Experimental results on CIFAR100, ImageNet-Full and ImageNet-Subset datasets show that multiple state-of-the-art CIL methods can be directly combined with our PKENet to reap significant performance improvement. Code: https://github.com/XXDyeah/PKENet.
Keywords:
Class-Incremental Learning
Potential knowledge extraction
Inter-sample relation
Exemplar selection
Debiased prediction
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers
Efficient continual learning in neural networks with embedding regularization.png)
NEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAIShareSaveShareSaveShareSaveShareSaveShareSave
ShareSave
View more