Coarse to fine-based image-point cloud fusion network for 3D object detection

Meilan Hao

doi:10.1016/j.inffus.2024.102551

Coarse to fine-based image-point cloud fusion network for 3D object detection

2024-12-01

6

PRE

AI

DOI：10.1016/j.inffus.2024.102551

Original

Original request for help

Save

Abstract

En 中文

Enhancing original LiDAR point cloud features with virtual points has gained widespread attention in multimodal information fusion. However, existing methods struggle to leverage image depth information due to the sparse nature of point clouds, hindering proper alignment with camera-derived features. We propose a novel 3D object detection method that refines virtual point clouds using a coarse-to-fine approach, incorporating a dynamic 2D Gaussian distribution for better matching and a dynamic posterior density-aware RoI network for refined feature extraction. Our method achieves an average precision (AP) of 90.02% for moderate car detection on the KITTI validation set, outperforming state-of-the-art methods. Additionally, our approach yields AP scores of 86.58% and 82.16% for moderate and hard car detection categories on the KITTI test set, respectively. These results underscore the effectiveness of our method in addressing point cloud sparsity and enhancing 3D object detection performance. The code is available at https://github.com/ZhongkangZ/LidarIG.

Keywords:

Coarse to fine

Image-point cloud fusion

Multimodal object detection

Dynamic 2D Gaussian distribution

Quantized perception strategy

Coarse to fine-based image-point cloud fusion network for 3D object detection

Abstract

Journal

Organization