.png){kind=icon}
返回
Dynamic Point-Pixel Feature Alignment for Multimodal 3-D Object Detection
.png){kind=icon}
PRE
AI
DOI：10.1109/JIOT.2023.3329884.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Detection of small or distant objects is a major challenge in 3-D object detection in autonomous driving either through RGB images or LiDAR point clouds. Despite the growing popularity of sensor fusion in this task, existing fusion methods have not adequately taken into account the challenges associated with 3-D small object detection, such as semantic misalignment of small objects, caused by occlusion and calibration errors. To address this issue, we propose dynamic point-pixel feature alignment network (DPPFA-Net) for multimodal 3-D small object detection by introducing memory-based point-pixel fusion (MPPF) modules, deformable point-pixel fusion (DPPF) modules, and semantic alignment evaluator (SAE) modules. More concretely, the proposed MPPF module automatically performs intramodal and cross-modal feature interactions. The intramodal interaction reduces sensitivity to noise points, while the explicit cross-modal feature interaction based on the memory bank facilitates easier network learning and enables a more comprehensive and discriminative feature representation. The DPPF module establishes interactions exclusively with key position pixels based on a sampling strategy. This design not only guarantees a low-computational complexity but also enables adaptive fusion functionality, especially beneficial for high-resolution images. The SAE module guarantees semantic alignment of the fused features, thereby enhancing the robustness and reliability of the fusion process. Furthermore, we construct a simulated multimodal noise data set, which enables quantitative analysis of the robustness of multimodal methods under varying degrees of multimodal noise. Extensive experiments on the KITTI benchmark and challenging multimodal noisy cases show that DPPFA-Net achieves a new state-of-the-art, highlighting its effectiveness in detecting small objects. Our proposed method is compared to the first place on the KITTI leaderboard and achieves better performance by 2.07%, 6.52%, 7.18%, and 6.22% of the average precision on the varying degrees of multimodal noise cases.
Keyword:
Three-dimensional displays
Object detection
Laser radar
Point cloud compression
Feature extraction
Semantics
Cameras
3-D object detection
autonomous driving
multimodal fusion
point clouds
期刊
IF:
8.9
论文数:
1.4W
被引数:
7.8W
机构
引用论文
.png){kind=icon}
Maximizing the Computational Efficiency of Temporary Housing Decision Support Following Disasters
PREAI
分享收藏
Pressure-Based Bioassay Perceived by a Flexible Pressure Sensor with Synergistic Enhancement of the Photothermal Effect通过具有协同增强光热效应的柔性压力传感器感知的基于压力的生物测定
PREAI
分享收藏分享收藏
查看更多内容