.png){kind=icon}
返回
Cross-Modality Gesture Recognition With Complete Representation Projection
.png){kind=icon}
PRE
AI
DOI：10.1109/JIOT.2024.3350692.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Human gesture recognition, due to its indispensable role in a myriad of emerging applications, has attracted close attention in the visual and wireless sensing community. The intrinsic characteristics of visual and wireless modalities are complementary to each other, e.g., wireless signals are robust to illumination changes and occluded conditions but suffer from low-spatial resolution, while visual signals have a high-spatial resolution but are vulnerable to scenario variations. Intuitively, integrating them has the potential chance to improve the overall discriminative ability. However, due to their different physical patterns and semantics, how to explore the relationship between the two modalities and leverage their complementary information to improve recognition performance still remains unsolved. In this article, we propose a complete representation projection method, which projects the signals from the heterogeneous modalities into a complete representation feature space by performing a bidirectional projection constraint. Furthermore, to relieve the low-projection efficiency problem caused by the heterogeneity of the two modality features, we propose an attention-based cross-modality interaction (ACMI) mechanism to perform implicit semantic feature alignment, thereby better capturing the complex dependencies between the two modalities and improving the feature projection efficiency. To evaluate the proposed method, we build a visual-radar cross-modality gesture recognition system and conduct extensive experiments. Experimental results demonstrate that the proposed approach not only performs favorably against vision-only and wireless-only solutions by a large margin, but also shows superiority over traditional fusion solutions.
Keyword:
Complete representation projection (CRP)
cross-modality
gesture recognition
visual
wireless
期刊
IF:
8.9
论文数:
1.4W
被引数:
7.8W
机构
引用论文
Assessing the impact of online postal self-sampling for sexually transmitted infections on health inequalities, access to care and clinical outcomes in the UK: protocol for ASSIST, a realist evaluation评估在线邮政自采样在性传播感染中的应用对英国健康不平等、医疗服务获取和临床结局的影响：ASSIST现实性评估方案.png)
BMJ Open
OAAI
BMJ OpenIF0
OAAI分享收藏分享收藏分享收藏Smart healthcare solutions using the internet of medical things for hand gesture recognition system
OAAI
OAAI分享收藏
查看更多内容