.png){kind=icon}
返回
Mitigating gradient conflicts via expert squads in multi-task learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.128832.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other stateof-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at https://github.com/chenjie04/Multi-Task-Learning-PyTorch.
Keyword:
Multi-task learning
Gradient conflicts
Expert squads
Task-specific knowledge
Shared features
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏Effect of intracerebroventricular insulin infusion on diabetic hyperphagia and hypothalamic neuropeptide gene expression.png)
Diabetes
DiabetesIF0
PREAI
分享收藏
查看更多内容