.png){kind=icon}
.png){kind=icon}
Return
RMAF: A replay method based on active forgetting for continual learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.131098.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The goal of continual learning (CL) is to enable deep neural networks to maintain good performance on old tasks while continuously learning new tasks. The main challenge in the field of CL is catastrophic forgetting (CF). This occurs because of the shared nature of model parameters, where learning a new task can overwrite or cause the loss of knowledge from previous tasks due to parameter updates. To alleviate this problem, the replay method is widely regarded as a simple and effective strategy that restores the performance of the model on the previous task by using a small amount of old task data. However, existing replay methods usually mix replay samples with new task data for training. This strategy makes it difficult to effectively adjust the shared parameters that cause CF, thereby weakening the effect of alleviating forgetting. To address the above problems, this paper proposes a replay method based on active forgetting (RMAF) under continual domain-adaptive pre-training (DAP-training). Unlike traditional methods, RMAF does not mix replay data with current domain data for training, but uses the replay data employed to alleviate CF to train the shared parameters that cause forgetting separately. In addition, to avoid excessive bias of shared parameters towards the current domain, RMAF introduces an active forgetting mechanism to ensure that each domain achieves balanced learning during parameter adjustment. Experimental results show that RMAF outperforms existing methods and achieves the best performance on multiple benchmark datasets.
Keywords:
continual learning
catastrophic forgetting
replay method
domain-adaptive pre-training
active forgetting