arrow
Return

The continuous learning algorithm with null space and eigenface-based orthogonal weight modification

delete2025-07-01
delete0
PRE
AI
L
Liao, Dingding
J
Junfeng Liu
J
Jun Zeng *
X
Xu, Shikang
DOI:10.1016/j.eswa.2025.127468delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
The challenge in continual learning is to enable deep neural networks to learn new knowledge in sequential task learning without forgetting previously acquired task knowledge. Orthogonal Weight Modification (OWM) is a significant continual learning algorithm proposed in recent years. The CEig-OWM algorithm is a continual learning algorithm that employs orthogonal weight modification based on eigenface input space representation. To enhance the stability of the CEig-OWM algorithm in retaining learned knowledge, this study introduces the null space of zero eigenvalue vectors into the calculation of the input space orthogonal matrix within the CEig-OWM algorithm, yielding the Null Space Eigenface-based Orthogonal Weight Modification (NEig-OWM) continual learning algorithm. Extensive comparative experiments on continual learning have been conducted across various datasets, and the experimental results demonstrate that the NEig-OWM algorithm exhibits the best continual learning performance among those with comparable parameter sizes.
Keywords:
Continual learning
Orthogonal weight modification
Null space
Eigenface

Journal

Expert Systems with Applications cover
Expert Systems with Applications
IF:
7.5
Papers:
3.0W
Citations:
10.2W

Organization

No organization information available
Cited Papers

Cited Papers

Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting
err2022-05-01
err16
PREAI
errZhang, Baosheng; Guo, Yuchen; Li, Yipeng; He, Yuwei; Wang, Haoqian; Dai, Qionghai
errShare
errSave
Continual learning of context-dependent processing in neural networks
err2019-08-09
err156
PREAI
errZeng, Guanxiong; Chen, Yang; Cui, Bo; Yu, Shan
errShare
errSave
Disturbance-immune weight sharing for neural architecture search
err2021-12-01
err12
errOAAI
errNiu, Shuaicheng; Wu, Jiaxiang; Zhang, Yifan; Guo, Yong; Zhao, Peilin; Huang, Junzhou; Tan, Mingkui
errShare
errSave
Overcoming catastrophic forgetting in neural networks
err2017-03-14
err3.7K
errOAAI
errKirkpatricka, James; Pascanu, Razvan; Rabinowitz, Neil; Veness, Joel; Desjardins, Guillaume; Rusu, Andrei A.; Milan, Kieran; Quan, John; Ramalho, Tiago; Grabska-Barwinska, Agnieszka; Hassabis, Demis; Clopath, Claudia; Kumaran, Dharshan; Hadsell, Raia
errShare
errSave
no more