.png){kind=icon}
返回
Task-Aware Dynamic Model Optimization for Multi-Task Learning
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3339793.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-task learning (MTL) is a field in which a deep neural network simultaneously learns knowledge from multiple tasks. However, achieving resource-efficient MTL remains challenging due to entangled network parameters across tasks and varying task-specific complexity. Existing methods employ network compression techniques while maintaining comparable performance, but they often compress uniformly across all tasks without considering individual complexity. This can lead to suboptimal solutions due to entangled network parameters and memory inefficiency, as the parameters for each task may be insufficient or excessive. To address these challenges, we propose a framework called Dynamic Model Optimization (DMO) that dynamically allocates network parameters to groups based on task-specific complexity. This framework consists of three key steps: measuring task similarity and task difficulty, grouping tasks, and allocating parameters. This process involves the calculation of both weight and loss similarities across tasks and employs sample-wise loss as a measure of task difficulty. Tasks are grouped based on their similarities, and parameters are allocated with dynamic pruning according to task difficulty within their respective groups. We apply the proposed framework to MTL with various classification datasets. Experimental results demonstrate that the proposed approach achieves high performance while taking fewer network parameters than other MTL methods.
Keyword:
Multi-task learning
resource-efficient learning
model optimization
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Coal resources under carbon peak: Segmentation of massive laser point clouds for coal mining in underground dusty environments using integrated graph deep learning model碳峰值下的煤炭资源: 基于集成图深度学习模型的地下尘土环境下煤炭开采的海量激光点云分割.png)
ENERGY
ENERGYIF9.4
PREAI
分享收藏
分享收藏Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition人与计算机: 交通标志识别的基准机器学习算法NEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏
查看更多内容