Dual-balancing for multi-task learning

doi:10.1016/j.neunet.2025.108317

Dual-balancing for multi-task learning

2025-11-11

0

PRE

AI

DOI：10.1016/j.neunet.2025.108317

Original

Original request for help

Save

Abstract

En 中文

Multi-task learning aims to learn multiple related tasks simultaneously and has achieved great success in various fields. However, the disparity in loss and gradient scales among tasks often leads to performance compromises, and the balancing of tasks remains a significant challenge. In this paper, we propose Dual-Balancing Multi-Task Learning (DB-MTL) to achieve task balancing from both the loss and gradient perspectives. Specifically, DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm. Extensive experiments on a number of benchmark datasets demonstrate that DB-MTL consistently performs better than the current state-of-the-art.

Dual-balancing for multi-task learning

Abstract

Journal

Organization