Knowledge Modeling and Distribution Refinement in Pre-trained Model for Class-Incremental Learning

Hae-Rin Byeon

doi:10.1007/978-3-032-10486-1_8

2026-01-01

PRE

Hae-Rin Byeon

Sung‐Bae Cho *

DOI：10.1007/978-3-032-10486-1_8

原文链接

摘要

En 中文

近期，类别增量学习利用大型预训练模型（PTMs）进行强大的特征提取。然而，现有方法仍面临两大关键挑战：（i）参数漂移，即在新类别上逐步微调权重会侵蚀先前建立的决策边界（灾难性遗忘），以及（ii）特征重叠，即新引入的类别嵌入与冻结PTM特征空间中的现有聚类发生碰撞。我们提出一种知识建模和分布精化（KMDR）方法以同时解决这两个问题。KMDR将每个类别分布编码为紧凑的高斯混合模型，作为固定大小的参数化记忆，无需存储原始样本。它进一步训练一个轻量级投影器，通过类内方差抑制损失来收紧聚类。该方法无需任务特定模块或复现缓冲区，却能通过长任务序列缓解参数漂移和特征重叠。在CIFAR-100、CUB-200、ImageNet-A和ImageNet-R基准数据集上的广泛评估表明，KMDR始终优于当前最优基线。

Keyword:

Class incremental learning

Continual learning

Gaussian mixture model

期刊

INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING-IDEAL 2025, PT I

IF:

论文数:

被引数:

机构

Yonsei University

学者数:

4.8W

论文数: 4.6W

被引数: 5.2W

引用论文

Learning to Prompt for Continual Learning

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2022-06-01

OAAI

Zifeng Wang; Zizhao Zhang; Chen-Yu Lee; Han Zhang; Ruoxi Sun; Xiaoqi Ren; Guolong Su; Vincent Perot; Jennifer Dy; Tomas Pfister

CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2023-06-01

OAAI

James Seale Smith; Leonid Karlinsky; Vyshnavi Gutta; Paola Cascante-Bonilla; Donghyun Kim; Assaf Arbelle; Rameswar Panda; Rogerio Feris; Zsolt Kira

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization 健壮性的多面: 分布外泛化的批判性分析

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

IF0

2021-10-01

OAAI

Dan Hendrycks; Steven Basart; Norman Mu; Saurav Kadavath; Frank Wang; Evan Dorundo; Rahul Desai; Tyler Zhu; Samyak Parajuli; Mike Guo; Dawn Song; Jacob Steinhardt; Justin Gilmer

A Comprehensive Survey of Continual Learning: Theory, Method and Application 持续学习的综合考察: 理论、方法与应用

IEEE Transactions on Pattern Analysis and Machine Intelligence

IF0

2024-08-01

OAAI

Liyuan Wang; Xingxing Zhang; Hang Su; Jun Zhu

Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2019-06-01