.png){kind=icon}
返回
Dual-mask: Progressively sparse multi-task architecture learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110950.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-task architecture learning has achieved significant success by learning optimal sharing architectures for different tasks. However, previous works to learn branched architectures for different tasks can sometimes lead to unsatisfying multi-task performance, as not all detailed branches are relevant to a specific task. Task-relevant architectures can be sparse, including only partial channels or layers in the entire architecture (i.e., a sub-network). In addition, most previous works rely on a heuristic architecture selection procedure that could not support continuous architecture optimization. To this end, in this paper, we propose dual-mask, a progressively sparse multi-task architecture learning method. Starting with a task-free architecture, it identifies the informative features along two-level, channels and layers, for each task, while suppressing conflicting or noisy parts in a differentiable manner, so that better task-specific sub-networks are captured. Specifically, the channel and layer selection modules produce respective hybrid binary and real value masks, designed to pick salient channels and layers for each task, respectively. To jointly optimize masks with model parameters, we propose an importance-guided relaxation method for solving the stochastic binary optimization problem, after which the interference or noise parts can be pruned by masks. Additionally, a progressive training strategy with continuation is provided that gradually sparsity the task-specific sub-networks. Experiments show that dual-mask achieves superior performance than SOTA multi-task methods.
Keyword:
Multi-task learning
Neural network
Sparse network
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Electrochemically assisted micro localized grafting of aptamers in a microchannel engraved in fluorinated thermoplastic polymer Dyneon THV.png)
RSC Advances
RSC AdvancesIF0
PREAI
分享收藏Damage Detection and Level Classification of Roof Damage After Typhoon Faxai Based on Aerial Photo and Deep Learning基于航拍照片和深度学习的Faxai台风后顶板损伤检测与等级分类
OAAI
IF0
OAAI分享收藏Multiprotocol Label Switching (MPLS) Traffic Engineering (TE) Management Information Base (MIB)
IF0
PREAI
分享收藏Efficient multi-task progressive learning for semantic segmentation and disparity estimation用于语义分割和视差估计的高效多任务渐进学习PATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAI分享收藏
查看更多内容