.png){kind=icon}
.png){kind=icon}
Return
Practical incremental learning: Striving for better performance-efficiency trade-off
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.129831.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We revisit the storage of historical data and buffer management in replay-based Incremental Learning (IL), where the buffer is usually a complete copy of the storage. Storage or buffer size constraints present significant challenges in laboratory settings but may not be practical in real-world applications. Since storing historical data is generally feasible and affordable in most realistic scenarios, we propose a generalization to IL by decoupling the concepts of storage and buffer, dubbed Practical Incremental Learning (PIL). We argue that, in practice, storage size should be determined by cost considerations, while buffer size should be limited for efficiency. Under PIL, we establish a strong baseline by resampling the buffer for each training iteration, ensuring better historical data coverage. We then evaluate popular strategies within the IL community, revealing their ineffectiveness against this baseline. We identify key limitations through in-depth analysis and derive insightful design principles for PIL methods. Based on these principles, Cross-Task Distance Calibration with Distribution Annealing (DCDA) is proposed. It achieves significant improvements over state-of-the-art methods across multiple benchmarks and settings, with performance gains of up to 15%, demonstrating its industrial-level performance and usability.
Keywords:
Incremental learning
Image recognition
Exemplar replay
Knowledge distillation
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W