.png){kind=icon}
Return
Dynamic Computation Allocation for Efficient Visual Object Tracking Guided by Runtime Scene Complexity
.png){kind=icon}
PRE
AI
DOI：10.1109/tcsvt.2026.3749787.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The computational efficiency of visual object tracking is of the utmost importance for practical deployment. The recent advances in visual object tracking focus on the tracker accuracy, as measured on standard benchmarks. However, these advances are invariably achieved by reliance on more complex models at the expense of a significant computational overhead. The high complexity severely restricts the real-time performance of tracking models, making them difficult to deploy on resource-constrained devices. To mitigate this trend, we propose an innovative tracking solution, that monitors the scene complexity in real time, and uses this information dynamically to control the allocation of computing resources to the backbone tracking network. We employ a simple multi-layer transformer backbone that performs feature extraction and template-search interaction. Our scene-complexity scoring module is supervised by runtime quality assessment performed by the early transformer layers, thereby estimating the difficulty of tracking the current frame. Based on the score, our framework can dynamically allocate computational resources to different transformer layers, reducing unnecessary computations in simple scenes and increasing the effort for tracking in complex ones. To ensure a wide range of applications, we provide three versions of our tracker: DCATrack-L12, DCATrack-L8 and DCATrack-L4. DCATrack-L12 achieves an AUC of 69.0% on LaSOT and an AUC of 49.4% on Lasotext, with a high GPU speed of 171 FPS, which surpasses our SOTA baseline by 55% FPS on GPU with comparable accuracy. DCATrack-L8 achieves an AUC of 67.9% on LaSOT with a higher GPU speed of 211 FPS. DCATrack-L4 surpasses MixFormerV2-S by 78 FPS on GPU and 14 FPS on CPU with a higher accuracy on mainstream datasets. The experimental results demonstrate that our lightweight model achieves a remarkable balance between computational efficiency and tracking performance, as well as providing, as a by-product, a measure of scene complexity for use in any downstream vision analysis. Codes and checkpoints are available at https://github.com/JNU2023ZY/DCATrack.
Keywords:
Visual object tracking
Transformer tracker
Real-time tracking
Dynamic computation allocation
Early exit
Journal
IF:
11.1
Papers:
1.8K
Citations:
3.1W
Organization
Cited Papers
No cited papers available