.png){kind=icon}
返回
Model Behavior Preserving for Class-Incremental Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2022.3144183.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep models have shown to be vulnerable to catastrophic forgetting, a phenomenon that the recognition performance on old data degrades when a pre-trained model is fine-tuned on new data. Knowledge distillation (KD) is a popular incremental approach to alleviate catastrophic forgetting. However, it usually fixes the absolute values of neural responses for isolated historical instances, without considering the intrinsic structure of the responses by a convolutional neural network (CNN) model. To overcome this limitation, we recognize the importance of the global property of the whole instance set and treat it as a behavior characteristic of a CNN model relevant to model incremental learning. On this basis: 1) we design an instance neighborhood-preserving (INP) loss to maintain the order of pair-wise instance similarities of the old model in the feature space; 2) we devise a label priority-preserving (LPP) loss to preserve the label ranking lists within instance-wise label probability vectors in the output space; and 3) we introduce an efficient derivable ranking algorithm for calculating the two loss functions. Extensive experiments conducted on CIFAR100 and ImageNet show that our approach achieves the state-of-the-art performance.
Keyword:
Data models
Adaptation models
Computational modeling
Training
Task analysis
Neural networks
Perturbation methods
Catastrophic forgetting
continual learning
continuous learning
incremental learning
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
.png){kind=icon}
The effect of maternal exercise on umbilical artery blood flow in pregnancy-induced hypertension
PREAI
分享收藏
查看更多内容