Hierarchical surrogate-based frame skip optimization for multi-object tracking

doi:10.1016/j.ins.2026.123610

Hierarchical surrogate-based frame skip optimization for multi-object tracking

2026-10-15

0

PRE

AI

DOI：10.1016/j.ins.2026.123610

Original

Original request for help

Save

Abstract

En 中文

Multi-Object Tracking (MOT) typically requires executing computationally intensive detection models on every frame, posing significant challenges in real-time and resource-constrained environments. To address this issue, we propose the Hierarchical Surrogate-Based Frame Skip Optimization (HSFSO) framework to adaptively reduce detection frequency while preserving a favorable trade-off between tracking accuracy and computational efficiency. HSFSO consists of two stages: (1) Frame Skip Ratio Optimization, which determines a globally suitable skip ratio based on image features, and (2) Motion-Based Frame Selection, which refines the decision at the frame level using optical flow. We integrate HSFSO with ByteTrack using a lightweight YOLOv8-small detector to target resource-constrained deployment scenarios. In addition, a multi-layer perceptron (MLP)-based surrogate model predicts tracking accuracy in real time, enabling dynamic skip-ratio adaptation without expensive online evaluation. Experiments on MOT17 and MOT20 show that HSFSO outperforms existing frame-skipping baselines while maintaining practical throughput gains. On MOT17, HSFSO attains a MOTA of 0.5358 at a = 0.15, and ablation studies confirm the complementary roles of the two hierarchical stages. These findings indicate that hierarchical, data-driven frame skipping is a promising approach for improving efficiency in resource-constrained MOT scenarios.

Keywords:

Multi-object tracking

Frame skipping optimization

Hierarchical learning

Video processing

Optical flow analysis

Computational efficiency

Hierarchical surrogate-based frame skip optimization for multi-object tracking

Abstract

Journal

Organization