.png){kind=icon}
返回
Multi-Adaptive Optimization for multi-task learning with deep neural networks
.png){kind=icon}
OA
AI
DOI：10.1016/j.neunet.2023.11.038.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-task learning is a promising paradigm to leverage task interrelations during the training of deep neural networks. A key challenge in the training of multi-task networks is to adequately balance the complementary supervisory signals of multiple tasks. In that regard, although several task-balancing approaches have been proposed, they are usually limited by the use of per-task weighting schemes and do not completely address the uneven contribution of the different tasks to the network training. In contrast to classical approaches, we propose a novel Multi-Adaptive Optimization (MAO) strategy that dynamically adjusts the contribution of each task to the training of each individual parameter in the network. This automatically produces a balanced learning across tasks and across parameters, throughout the whole training and for any number of tasks. To validate our proposal, we perform comparative experiments on real-world datasets for computer vision, considering different experimental settings. These experiments allow us to analyze the performance obtained in several multi-task scenarios along with the learning balance across tasks, network layers and training steps. The results demonstrate that MAO outperforms previous task-balancing alternatives. Additionally, the performed analyses provide insights that allow us to comprehend the advantages of this novel approach for multi-task learning.
Keyword:
Multi-task learning
Computer vision
Neural networks
Deep learning
Optimization
Gradient descent
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
TSFD-Net: Tissue specific feature distillation network for nuclei segmentation and classification.png)
NEURAL NETWORKSOAAI
NEURAL NETWORKSIF6.3
OAAI分享收藏SelfVIO: Self-supervised deep monocular Visual-Inertial Odometry and depth estimationNEURAL NETWORKSOAAI
NEURAL NETWORKSIF6.3
OAAI分享收藏Effect of intracerebroventricular insulin infusion on diabetic hyperphagia and hypothalamic neuropeptide gene expressionDiabetes
DiabetesIF0
PREAI
分享收藏Renal Tubular Secretion of Tanshinol: Molecular Mechanisms, Impact on Its Systemic Exposure, and Propensity for Dose-Related Nephrotoxicity and for Renal Herb-Drug Interactions丹参素的肾小管分泌: 分子机制，对其全身暴露的影响以及剂量相关的肾毒性和肾草药相互作用的倾向OAAI
OAAI分享收藏分享收藏
查看更多内容