Rethinking softmax in incremental learning

doi:10.1016/j.neunet.2025.108017

Rethinking softmax in incremental learning

2025-08-20

0

PRE

AI

DOI：10.1016/j.neunet.2025.108017

Original

Original request for help

Save

Abstract

En 中文

• The softmax cross-entropy is a widely used loss function for supervised learning problems. • The classical Learning Without Forgetting (LWF) framework is a popular incremental learning strategy that simultaneously optimizes the softmax cross-entropy and knowledge distillation loss for incremental learning tasks. • The use of softmax cross-entropy loss in incremental learning tasks may lead to non-identifiability issues, which in turn can introduce weighting imbalances across tasks. • By addressing the non-identifiability issues, we can improve prediction accuracy and mitigate the phenomenon of forgetting.

Keywords:

Continual learning

Catastrophic forgetting

Distillation loss

Life-long learning

Incremental learning

Rethinking softmax in incremental learning

Abstract

Journal

Organization