.png){kind=icon}
返回
Linguistic-Driven Partial Semantic Relevance Learning for Skeleton-Based Action Recognition
.png){kind=icon}
OA
AI
DOI：10.3390/s24154860.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Skeleton-based action recognition, renowned for its computational efficiency and indifference to lighting variations, has become a focal point in the realm of motion analysis. However, most current methods typically only extract global skeleton features, overlooking the potential semantic relationships among various partial limb motions. For instance, the subtle differences between actions such as brush teeth and brush hair are mainly distinguished by specific elements. Although combining limb movements provides a more holistic representation of an action, relying solely on skeleton points proves inadequate for capturing these nuances. Therefore, integrating detailed linguistic descriptions into the learning process of skeleton features is essential. This motivates us to explore integrating fine-grained language descriptions into the learning process of skeleton features to capture more discriminative skeleton behavior representations. To this end, we introduce a new Linguistic-Driven Partial Semantic Relevance Learning framework (LPSR) in this work. While using state-of-the-art large language models to generate linguistic descriptions of local limb motions and further constrain the learning of local motions, we also aggregate global skeleton point representations and textual representations (which generated from an LLM) to obtain a more generalized cross-modal behavioral representation. On this basis, we propose a cyclic attentional interaction module to model the implicit correlations between partial limb motions. Numerous ablation experiments demonstrate the effectiveness of the method proposed in this paper, and our method also obtains state-of-the-art results.
Keyword:
skeleton-based action recognition
cross-modal
transformer
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.5
论文数:
7.2W
被引数:
20.9W
机构
引用论文
Evaluation of the Effect of Systolic Blood Pressure and Pulse Pressure on Cognitive Function: The Women's Health and Aging Study II.png)
PLoS ONEOAAI
PLoS ONEIF0
OAAI分享收藏Social and cultural barriers to husbands� involvement in maternal health in rural, Gambia农村地区丈夫参与孕产妇健康的社交与文化障碍，冈比亚OAAI
OAAI分享收藏Metal-organic chemical vapor deposition of indium selenide films using a single-source precursor使用单源前体对硒化铟薄膜进行金属有机化学气相沉积
PREAI
分享收藏
查看更多内容