.png){kind=icon}
Return
A scalable complexity-regularized modular network for continual learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2026.135079.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Modular continual learning supports knowledge reuse and capacity adaptation by composing existing modules and introducing new computational units when needed. However, dynamically expandable modular models must coordinate two coupled processes: composing the current modules effectively and determining when their representational capacity is insufficient for an incoming task. To address these challenges, we propose Complexity-Regularized Differentiable Modular Search (CR-DMS), which integrates differentiable module composition with representation-driven structural expansion. CR-DMS introduces a Layer-wise Structural Complexity (LSC) objective that regularizes the dispersion of module contribution distributions and the pairwise similarity of module representations during differentiable modular search. A Neural Control Network (NCN) evaluates the representational adequacy of existing modules and triggers layer-wise expansion when a representation gap is detected. Newly introduced modules are optimized through a dedicated training stage before being integrated into the modular search space. Experiments are conducted on five heterogeneous task streams from the CTrL benchmark and on task-incremental partitions of CIFAR-100 and TinyImageNet-200. On CTrL, CR-DMS achieves the numerically highest mean final accuracy, zero signed forgetting matching several modular baselines, and the second-highest mean transfer score among the compared methods. It also maintains competitive classification performance across different task partitions while requiring fewer average FLOPs and shorter training time than the expandable modular baseline LMC. Ablation studies show that both the overall LSC objective and the dedicated training stage contribute consistently to final classification performance.
Keywords:
Continual learning
Modular networks
Catastrophic forgetting
Dynamically scalable networks
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
View more organizations
Cited Papers
No cited papers available