.png){kind=icon}
Return
Trainable Weights for Multitask Learning
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3319072.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The research on multi-task learning has been steadily increasing due to its advantages, such as preventing overfitting, averting catastrophic forgetting, solving multiple inseparable tasks, and coping with data shortage. Here, we question whether to incorporate different orderings of feature levels based on distinct characteristics of tasks and their interrelationships in multitask learning. While in many classification tasks, leveraging the features extracted from the last layer is common, we thought that given different characteristics of tasks there might be a need to encompass different representation levels, i.e., different orderings of feature levels. Hence, we utilized the knowledge of different representation levels by features extracted from the various blocks of the main module and applied trainable parameters as weights on the features. This indicates that we optimized the solution to the question by learning to weigh the features in a task-specific manner and solving tasks with a combination of newly weighted features. Our method SimPara presents a modular topology of multitask learning that is efficient in terms of memory and computation, effective, and easily applicable to diverse tasks or models. To show that our approach is task-agnostic and highly applicable, we demonstrate its effectiveness in auxiliary task learning, active learning, and multilabel learning settings. This work underscores that by simply learning weights to better order the features learned by a single backbone, we can incur better task-specific performance of the model.
Keywords:
Auxiliary task learning
incremental learning multitask learning
trainable parameters
Journal
IF:
3.6
Papers:
9.8W
Citations:
29.4W
Organization
View more organizations
Cited Papers
Rupture Process of the Mw 3.3 Earthquake in the St. Gallen 2013 Geothermal Reservoir, Switzerland
OAAI
.png){kind=icon}
OAAIShareSaveShareSaveShareSaveShareSaveShareSaveShareSaveShareSave View more