Online knowledge distillation optimization based on Multi-Student model Multi-Task collaborative learning

doi:10.1016/j.knosys.2026.115910

Online knowledge distillation optimization based on Multi-Student model Multi-Task collaborative learning

2026-04-02

0

PRE

AI

DOI：10.1016/j.knosys.2026.115910

Original

Original request for help

Save

Abstract

En 中文

• The MTCL framework is proposed to address representation homogeneity via collaborative learning. • Heterogeneous auxiliary tasks are introduced to construct complementary Geometric and Semantic Experts. • A Collaborative Processing Module (CPM) integrates diverse features without rigid alignment. • The method achieves a significant 11.19% accuracy gain on TinyImageNet using ResNet-34. • Cost-effectiveness analysis demonstrates a high return on investment (ROI) with standard budgets.

Keywords:

Multi-Task Learning

Collaborative Learning

Knowledge Distillation

Representation Homogeneity

Cost-Effectiveness

Online knowledge distillation optimization based on Multi-Student model Multi-Task collaborative learning

Abstract

Journal

Organization