.png){kind=icon}
Return
A Dynamic Feature Interaction Framework for Multi-task Visual Perception
.png){kind=icon}
OA
AI
DOI：10.1007/s11263-023-01835-5.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multi-task visual perception has a wide range of applications in scene understanding such as autonomous driving. In this work, we devise an efficient unified framework to solve multiple common perception tasks, including instance segmentation, semantic segmentation, monocular 3D detection, and depth estimation. Simply sharing the same visual feature representations for these tasks impairs the performance of tasks, while independent task-specific feature extractors lead to parameter redundancy and latency. Thus, we design two feature-merge branches to learn feature basis, which can be useful to, and thus shared by, multiple perception tasks. Then, each task takes the corresponding feature basis as the input of the prediction task head to fulfill a specific task. In particular, one feature merge branch is designed for instance-level recognition the other for dense predictions. To enhance inter-branch communication, the instance branch passes pixel-wise spatial information of each instance to the dense branch using efficient dynamic convolution weighting. Moreover, a simple but effective dynamic routing mechanism is proposed to isolate task-specific features and leverage common properties among tasks. Our proposed framework, termed D2BNet, demonstrates a unique approach to parameter-efficient predictions for multi-task perception. In addition, as tasks benefit from co-training with each other, our solution achieves on par results on partially labeled settings on nuScenes and outperforms previous works for 3D detection and depth estimation on the Cityscapes dataset with full supervision.
Keywords:
Multi-task perception
Dynamic routing
3D object detection
Panoptic segmentation
Depth estimation
Journal
IF:
9.3
Papers:
3.9K
Citations:
2.8W
Organization
View more organizations
Cited Papers
Continuous exposure to 900MHz GSM-modulated EMF alters morphological maturation of neural cells
PREAI
ShareSaveShareSaveShareSaveShareSaveShareSaveShareSaveShareSaveEnergy-Efficient Robot Configuration and Motion Planning Using Genetic Algorithm and Particle Swarm Optimization.png)
EnergiesOAAI
EnergiesIF0
OAAIShareSaveSystematic review of systematic reviews of non-pharmacological interventions to treat behavioural disturbances in older patients with dementia. The SENATOR-OnTop seriesBMJ Open
OAAI
BMJ OpenIF0
OAAIShareSave View more