.png){kind=icon}
返回
Instance-Aware Monocular 3D Semantic Scene Completion
.png){kind=icon}
PRE
AI
DOI：10.1109/TITS.2023.3344806.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We study outdoor 3D scene understanding, a challenging task demanding the intelligent system to infer both geometry and semantics from a single-view image - a critical skill for autonomous vehicles to navigate in the real 3D world. Towards this end, we present an instance-aware monocular semantic scene completion framework. To the best of our knowledge, this is the first endeavor specifically targeting the challenge of instance perception in the camera-based semantic scene completion task. Our method consists of two stages. In stage I, we design a region-based VQ-VAE network, providing an effective solution for 3D occupancy prediction. In stage II, we first introduce an instance-aware attention module, explicitly incorporating instance-level cues captured from mask images to enhance the instance features in RGB images. Then we leverage the deformable cross-attention to aggregate image features corresponding to each voxel query and utilize the deformable self-attention to refine query proposals. We combine these key ingredients and evaluate our method on two challenging datasets, namely SemanticKITTI and SSCBench-KITTI-360. The results unequivocally demonstrate the superiority of our proposed method over the state-of-the-art VoxFormer-S. Specifically, our method surpasses VoxFormer-S by 0.22 IoU and 0.72 mIoU on the validation set and achieves an impressive improvement of 3.04 IoU and 1.06 mIoU on the SSCBench-KITTI-360 validation set. Meanwhile, our approach ensures accurate perception of critical instances, thereby exhibiting its exceptional performance and potential for practical deployment.
Keyword:
3D scene understanding
semantic scene completion
3D vision
期刊
IF:
8.4
论文数:
9.7K
被引数:
6.3W
机构
引用论文
DcTr: Noise-robust point cloud completion by dual-channel transformer with cross-attentionDcTr: 通过具有交叉注意的双通道变压器完成噪声鲁棒的点云.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review自动驾驶中图像和点云融合的深度学习: 综述
OAAI
OAAI分享收藏分享收藏
查看更多内容