.png){kind=icon}
返回
Focal Channel Knowledge Distillation for Multi-Modality Action Recognition
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3298647.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The multi-modality action recognition aims to learn the complementary information from multiple modalities to improve the action recognition performance. However, there exists a significant modality channel difference, the equal transferring channel semantic features from multi-modalities to RGB will result in competition and redundancy during knowledge distillation. To address this issue, we propose a focal channel knowledge distillation strategy to transfer the key semantic correlations and distributions of multi-modality teachers into the RGB student network. The focal channel correlations provide intrinsic relationships and diversity properties of key semantics, and focal channel distributions provide salient channel activation of features. By ignoring the less-discriminative and irrelevant channels, the student can more efficiently utilize the channel capability to learn the complementary semantic features from the other modalities. Our focal channel knowledge distillation achieves 91.2%, 95.6%, 98.3% and 81.0% accuracy with 4.5%, 4.2%, 3.7% and 7.1% improvement on NTU 60 (CS), UTD-MHAD, N-UCLA and HMDB51 datasets comparing to unimodal RGB models. This focal channel knowledge distillation framework can also be integrated with the unimodal models to achieve the state-of-the-art performance. The extensive experiments show that the proposed method achieves 92.5%, 96.0%, 98.9%, and 82.3% accuracy on NTU 60 (CS), UTD-MHAD, N-UCLA, and HMDB51 datasets respectively.
Keyword:
Action recognition
knowledge distillation
multi-modality
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Learning 3D Skeletal Representation From Transformer for Action Recognition从Transformer学习3D骨骼表示以进行动作识别.png)
IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏分享收藏分享收藏
查看更多内容