.png){kind=icon}
.png){kind=icon}
Return
DIMTrack: Dynamic Neuron-Based RGBX Tracking for Multimodal Visual Cross-Modal Interaction
.png){kind=icon}
PRE
AI
DOI：10.1016/j.dsp.2025.105827.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal sensors are invaluable in visual tracking tasks due to their distinct advantages in addressing various challenging scenarios. Although it would be ideal to utilize a common model for all modalities, practical applications often necessitate the use of a single modality due to data scarcity. This study first proposes a multimodal tracking method based on the Dynamic Interaction Module (DIM), which aims to enhance tracking performance by dynamically fusing the feature information from RGB and X modalities. During the mixed training process, the DIM module facilitates the matching of similarities between modalities and provides a unified training framework for diverse modalities. Additionally, the DIM module employs expert models (including Avg Expert and Mix Expert) to balance the feature representations of various modalities, thereby ensuring feature optimization and enhancing model performance. Through extensive experiments with paired modalities such as RGB-E, RGB-D, and RGB-T, we demonstrate that the proposed method outperforms the RGB-X tracker during the inference process. Dynamic neurons optimize the fusion of cross-modal features by selectively focusing on the most relevant modal features, further enhancing the model’s robustness and tracking accuracy. Experimental results indicate that the proposed method significantly enhances performance in multimodal tracking tasks and demonstrates exceptional effectiveness and flexibility in complex scenarios.
Journal
D
IF:
3
Papers:
653
Citations:
0