.png){kind=icon}
返回
A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TCYB.2022.3223377.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multitask learning (MTL) is a powerful technique for jointly learning multiple tasks. However, it is difficult to achieve a tradeoff between tasks during iterative training, as some tasks may compete with each other. Existing methods manually design specific network models to mitigate task conflicts, but they require considerable manual effort and prior knowledge about task relationships to tune the model so as to obtain the best performance for each task. Moreover, few works have offered formal descriptions of task conflicts and theoretical explanations for the cause of task conflict problems. In this article, we provide a formal description of task conflicts that are caused by the gradient interference problem of tasks. To alleviate this issue, we propose a novel model-agnostic approach to mitigate gradient interference (MAMG) by designing a gradient clipping rule that directly modifies the interfering components on the gradient interfering direction. Specifically, MAMG is model-agnostic and thus it can be applied to a large number of multitask models. We also theoretically prove the convergence of MAMG and its superiority to existing MTL methods. We evaluate our method on a variety of real-world large datasets, and extensive experimental results confirm that MAMG can outperform some state-of-the-art algorithms on different types of tasks and can be easily applied to various methods.
Keyword:
Deep learning
gradient interference
multitask learning
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
Therapeutic response to prednisone in relation to age in polymyalgia rheumatica: a comparison study
.png){kind=icon}
PREAI
分享收藏分享收藏An Effective Knowledge Transfer Approach for Multiobjective Multitasking Optimization多目标多任务优化的有效知识转移方法
PREAI
分享收藏Multiclass Classification With Fuzzy-Feature Observations: Theory and Algorithms具有模糊特征观察的多类分类: 理论和算法
OAAI
OAAI分享收藏
查看更多内容