FoCL: Feature-oriented continual learning for generative models

Qicheng Lao

doi:10.1016/j.patcog.2021.108127

FoCL: Feature-oriented continual learning for generative models

2021-12-01

9

OA

AI

DOI：10.1016/j.patcog.2021.108127

Original

Save

View PDF

Abstract

En 中文

In this paper, we propose a general framework in continual learning for generative models: Feature -oriented Continual Learning (FoCL). Unlike previous works that aim to solve the catastrophic forgetting problem by introducing regularization in the parameter space or image space, FoCL imposes regulariza-tion in the feature space. We show in our experiments that FoCL has faster adaptation to distributional changes in sequentially arriving tasks, and achieves state-of-the-art performance for generative models in task incremental learning. We discuss choices of combined regularization spaces towards different use case scenarios for boosted performance, e.g., tasks that have high variability in the background. Finally, we introduce a forgetfulness measure that fairly evaluates the degree to which a model suffers from for -getting. Interestingly, the analysis of our proposed forgetfulness score also implies that FoCL tends to have a mitigated forgetting for future tasks. (c) 2021 Elsevier Ltd. All rights reserved.

Keywords:

Catastrophic forgetting

Continual learning

Generative models

Feature matching

Generative replay

Pseudo-rehearsal

FoCL: Feature-oriented continual learning for generative models

Abstract

AI Summary

Journal

Organization