.png){kind=icon}
返回
Collaborative Multimodal Fusion Network for Multiagent Perception
.png){kind=icon}
PRE
AI
DOI：10.1109/TCYB.2024.3491756.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
With the increasing popularity of autonomous driving systems and their applications in complex transportation scenarios, collaborative perception among multiple intelligent agents has become an important research direction. Existing single-agent multimodal fusion approaches are limited by their inability to leverage additional sensory data from nearby agents. In this article, we present the collaborative multimodal fusion network (CMMFNet) for distributed perception in multiagent systems. CMMFNet first extracts modality-specific features from LiDAR point clouds and camera images for each agent using dual-stream neural networks. To overcome the ambiguity in-depth prediction, we introduce a collaborative depth supervision module that projects dense fused point clouds onto image planes to generate more accurate depth ground truths. We then present modality-aware fusion strategies to aggregate homogeneous features across agents while preserving their distinctive properties. To align heterogeneous LiDAR and camera features, we introduce a modality consistency learning method. Finally, a transformer-based fusion module dynamically captures cross-modal correlations to produce a unified representation. Comprehensive evaluations on two extensive multiagent perception datasets, OPV2V and V2XSet, affirm the superiority of CMMFNet in detection performance, establishing a new benchmark in the field.
Keyword:
Collaboration
Sensors
Laser radar
Cameras
Feature extraction
Three-dimensional displays
Point cloud compression
Autonomous vehicles
Sensor fusion
Sensor phenomena and characterization
3-D object detection
autonomous driving
collaborative perception
multiagent system
multimodal fusion
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
TLR4 endogenous ligand MRP8/14 level in enthesitis-related arthritis and its association with disease activity and TLR4 expression.png)
Rheumatology
OAAI
RheumatologyIF0
OAAI分享收藏Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous DrivingV2X通信辅助自动驾驶的中断感知协同感知
OAAI
OAAI分享收藏分享收藏Materializing Strategy: The Blurry Line between Strategy Formulation and Strategy Implementation
PREAI
分享收藏
查看更多内容