.png){kind=icon}
Return
Sample-level weighting for multi-task learning with auxiliary tasks
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-05300-9.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multi-task learning (MTL) can improve the generalization performance of neural networks by sharing representations with related tasks. Nonetheless, MTL is challenging in practice because it can also degrade performance through harmful interference between tasks. Recent work has pursued task-specific loss weighting as a solution for this interference. However, existing algorithms treat tasks as atomic, lacking the ability to explicitly separate harmful and helpful signals beyond the task level. To this end, we propose SLGrad, a sample-level highly-dynamic weighting algorithm for multi-task learning with auxiliary tasks. By exploiting a hold-out meta objective, SLGrad reshapes the task distributions through sample-level weights to eliminate harmful auxiliary signals and augment useful task signals. Substantial generalization performance gains are observed on both semi-synthetic and synthetic datasets, as well as on three common real-world supervised multi-task problems. By studying the resulting sample and task weight distributions, we show that SLGrad is able to bridge the existing gap between sample weighting in single-task learning and dynamic task weighting in multi-task learning.
Keywords:
Deep learning
Multi-task learning
Dynamic task weighting
Sample-level weighting
Journal
IF:
3.5
Papers:
7.6K
Citations:
1.7W
Organization
Cited Papers
HydaLearn Highly Dynamic Task Weighting for Multitask Learning with Auxiliary Tasks.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
ShareSaveShareSaveShareSaveShareSaveShareSave
View more