.png){kind=icon}
返回
SCREAM: Knowledge sharing and compact representation for class incremental learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.ipm.2023.103629.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Methods based on dynamic structures are effective in addressing catastrophic forgetting on Class-incremental learning (CIL). However, they often isolate sub-networks and overlook the integration of overall information, resulting in a performance decline. To overcome this limitation, we recognize the importance of knowledge sharing among sub-networks. On the basis of dynamic network, we established a novel two-stage CIL method called SCREAM that includes an Expandable Network (EN) Learning Stage and a Compact Representation (CR) Stage: (1) design a clustering loss function for EN, aggregating related instances and promoting information sharing; (2) design dynamic weight alignment to alleviate the classifier's bias towards new class knowledge; and (3) design a balanced decoupled distillation for CR, mitigating the impact of the long-tail effect during multiple compressions. To validate the performance of SCREAM, we use 3 widely used datasets and set different Buffersize (replay-buffer) for comparison with the current state-of-the-art models.The result show that on CIFAR-100 ImageNet-100/1000 and Tiny-ImageNet achieve an average accuracy exceeding 2.46%, 1.22% and 1.52%, respectively. When using a smaller buffersize, SCREAM also achieves an average accuracy exceeding 4.60%. Furthermore, SCREAM shows good performance in terms of Resources needed.
Keyword:
Incremental learning
Knowledge sharing
Rehearsal
Knowledge distillation
期刊
I
IF:
6.9
论文数:
5.2K
被引数:
1.4W
机构
引用论文
Precipitation Versus Partitioning Kinetics during the Quenching of Low-Carbon Martensitic Steels.png)
Metals
OAAI
MetalsIF0
OAAI分享收藏Investigation of Ti mesh-supported anodes for direct borohydride fuel cells直接硼氢化物燃料电池用钛网支撑阳极的研究
PREAI
分享收藏分享收藏
查看更多内容