.png){kind=icon}
.png){kind=icon}
Return
Opportunistic Dynamic Architecture for Class-Incremental Learning
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2025.3556471.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Continual learning has attracted increasing attention over the last few years, as it enables to continually learn new tasks over time, which has significant implication to many real-world applications. A large number of continual learning techniques are proposed and achieve promising performance; however, many of them commit to a fixed, large architecture at the beginning, which can waste the memory space and incur high training cost. To directly tackle this challenge, we propose an Opportunistic Dynamic Architecture, ODA, based on mixture of experts. ODA can automatically grow with more experts for new incoming tasks and opportunistically shrink by merging experts with similar weights. We evaluated ODA on three commonly used datasets: CIFAR-100, CUB, and iNaturalist, and compared against eight existing continual learning techniques. ODA not only outperforms these techniques but does so with a parameter size that is slightly smaller on average, maintaining memory efficiency without compromising accuracy. Furthermore, ODA achieves this with only around 16% of the training time across all datasets when updating for each new task, making it a highly resource-efficient solution for continual learning applications.
Keywords:
Continuing education
Computer architecture
Neurons
Accuracy
Training
Memory management
Vehicle dynamics
Deep learning
Toy manufacturing industry
Representation learning
Class-incremental learning
continual learning
mixture of expert