.png){kind=icon}
返回
PVConvNet: Pixel-Voxel Sparse Convolution for multimodal 3D object detection
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110284.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Current LiDAR-only 3D detection methods inevitably suffer from the sparsity of point clouds and insufficient semantic information. To alleviate this difficulty, recent proposals densify LiDAR points by depth completion and then perform feature fusion with image pixels at the data -level or result -level. However, these methods often suffer from poor fusion effects and insufficient use of image information for voxel feature -level fusion. Meanwhile, noises brought by inaccurate depth completion significantly degrade detection accuracy. In this paper, we propose PVConvNet, a unified framework for multi -modal feature fusion that cleverly combines LiDAR points, virtual points and image pixels. Firstly, we develop an efficient Pixel-Voxel Sparse Convolution (PVConv) to perform voxel-wise feature -level fusion of point clouds and images. Secondly, we design a Noise -Resistant Dilated Sparse Convolution (NRDConv) to encode the voxel features of virtual points, which effectively reduces the impact of noise. Finally, we propose a unified RoI pooling strategy, namely Multimodal Voxel-RoI Pooling, for improving proposal refinement accuracy. We evaluate PVConvNet on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that our method outperforms state-of-the-art multi -modal based methods, achieving a moderate 3D AP of 86.92% on the KITTI test set.
Keyword:
3D object detection
LiDAR points
Virtual points
Image pixels
Multi-modal fusion
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Objformer: Boosting 3D object detection via instance-wise interactionObjformer: 通过实例交互增强3D对象检测.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Materializing Strategy: The Blurry Line between Strategy Formulation and Strategy Implementation
PREAI
分享收藏Pressure-Based Bioassay Perceived by a Flexible Pressure Sensor with Synergistic Enhancement of the Photothermal Effect通过具有协同增强光热效应的柔性压力传感器感知的基于压力的生物测定
PREAI
分享收藏Spatial information enhancement network for 3D object detection from point cloud基于空间信息增强网络的点云三维目标检测PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏
查看更多内容