Multi-objective adaptive Nesterov-like acceleration algorithm for multi-task deep learning

Fanyun Meng

doi:10.1016/j.neucom.2025.131251

Multi-objective adaptive Nesterov-like acceleration algorithm for multi-task deep learning

2025-09-04

0

PRE

AI

DOI：10.1016/j.neucom.2025.131251

Original

Original request for help

Save

Abstract

En 中文

With the wide application of multi-task learning in many deep learning problems, how to balance different tasks has become the most important challenge in multi-task learning training. This paper proposes a fast multi-objective adaptive momentum algorithm to address the multi-objective characteristics of multi-task learning and the slow descent of adaptive algorithms. Firstly, the algorithm adopts multi-objective adaptive technology to separate the cumulative gradient of each parameter in the adaptive algorithm, thereby separating the learning rate of each target task. Secondly, a Nesterov-like momentum acceleration is added to each task-level gradient, and it avoids the additional overhead of Nesterov acceleration algorithm in calculating gradients at extrapolation points. For the task-dominated problem, we describe the task weights by using the relative loss of the task and the normalized gradient paradigm, and normalize them, which can effectively balance each task. Finally, theoretically, the descending boundary of the multi-task aggregation loss function is characterized. The experimental results show that, compared with usual adaptive algorithms, the proposed algorithm has higher performance and faster convergence in training the scene parsing multi-task learning problems.

Multi-objective adaptive Nesterov-like acceleration algorithm for multi-task deep learning

Abstract

Journal

Organization

Cited Papers