Multi-stage information diffusion for joint depth and surface normal estimation

Zhiheng Fu

doi:10.1016/j.patcog.2023.109660

Multi-stage information diffusion for joint depth and surface normal estimation

2023-09-01

6

OA

AI

DOI：10.1016/j.patcog.2023.109660

原文链接

查看原文

摘要

En 中文

Depth and surface normal estimations are important for 3D geometric perception, which has numer-ous applications including autonomous vehicles and robots. In this paper, we propose a lightweight Multi-stage Information Diffusion Network (MIDNet) for the simultaneous prediction of depth and sur-face normals from a single RGB image. To obtain semantic and detail-preserving features, we adopt a high-resolution network as our backbone to learn multi-scale features, which are then fused into shared features for the two tasks. To mutually boost each task, a Cross-Correlation Attention Module (CCAM) is proposed to adaptively integrate information for the prediction of the two tasks in multiple stages, in-cluding feature-level information interaction and task-level information interaction. Ablation studies show that the proposed multi-stage information diffusion strategy can boost the performance gain for the two tasks at different levels. Compared to current state-of-the-art methods on the NYU Depth V2, Stanford 2D-3D-Semantic and KITTI datasets, our method achieves superior performance for both monocular depth and surface normal estimation. (c) 2023 Elsevier Ltd. All rights reserved.

Keyword:

Depth estimation

Surface normal estimation

Multitask learning

Attention map

Multi -Stage information fusion

Multi-stage information diffusion for joint depth and surface normal estimation

摘要

AI总结

期刊

机构