.png){kind=icon}
返回
Sample-level weighting for multi-task learning with auxiliary tasks
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-05300-9.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-task learning (MTL) can improve the generalization performance of neural networks by sharing representations with related tasks. Nonetheless, MTL is challenging in practice because it can also degrade performance through harmful interference between tasks. Recent work has pursued task-specific loss weighting as a solution for this interference. However, existing algorithms treat tasks as atomic, lacking the ability to explicitly separate harmful and helpful signals beyond the task level. To this end, we propose SLGrad, a sample-level highly-dynamic weighting algorithm for multi-task learning with auxiliary tasks. By exploiting a hold-out meta objective, SLGrad reshapes the task distributions through sample-level weights to eliminate harmful auxiliary signals and augment useful task signals. Substantial generalization performance gains are observed on both semi-synthetic and synthetic datasets, as well as on three common real-world supervised multi-task problems. By studying the resulting sample and task weight distributions, we show that SLGrad is able to bridge the existing gap between sample weighting in single-task learning and dynamic task weighting in multi-task learning.
Keyword:
Deep learning
Multi-task learning
Dynamic task weighting
Sample-level weighting
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
引用论文
HydaLearn Highly Dynamic Task Weighting for Multitask Learning with Auxiliary Tasks.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
分享收藏Distant regulatory elements in a Sox10‐βGEO BAC transgene are required for expression of Sox10 in the enteric nervous system and other neural crest‐derived tissuesSox10-βgeo BAC转基因中的远距离调控元件是肠神经系统和其他神经源性组织中 Sox10 表达所必需的
OAAI
OAAI分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
查看更多内容