EMPC: Efficient multi-view parallel co-learning for semi-supervised action recognition

Anyang Tong

doi:10.1016/j.eswa.2024.124634

2024-12-01

PRE

Anyang Tong

Chao Tang *

DOI：10.1016/j.eswa.2024.124634

原文链接

原文求助

摘要

En 中文

Semi-supervised learning (SSL) is an effective approach to address the challenge of limited labeled data in action recognition. Existing methods have explored temporal augmentation and consistent learning, which have received widespread attention. However, these methods come with an exponential increase in computational effort, and neglect the potential for divergence and collaboration between modalities. Additionally, the models may exhibit randomness during pseudo-label evaluation and inconsistency between training and inference. To address these challenges, we propose an efficient multi-view parallel co-learning (EMPC) method for semisupervised action recognition. First, we explore the temporal gradient (TG) and create a new view that contains rich motion history information, called the historical temporal gradient (HTG). Second, inspired by the working mechanism of Dropout, we assemble a low-computational multi-functional committee (MFC) and perform pseudo-label editing based on two evaluation criteria: confidence and consistency. We further design a new regularization strategy based on MFC, called mean regularized dropout (MR-Drop), which measures and reduces the output distribution's uncertainty between sub-models to improve the model's performance. Finally, based on the complementary information between RGB and HTG views, we build an efficient parallel network with multi-view feature sharing and pseudo-label collaboration. We evaluate EMPC on three public datasets: UCF-101, HMDB-51, and Kinetics-100. The experimental results demonstrate that EMPC achieves better classification performance with a limited amount of labeled data and a large amount of unlabeled data.

Keyword:

Action recognition

Semi-supervised learning

Temporal gradient

Co-learning

Dropout