.png){kind=icon}
返回
SML: A Skeleton-based multi-feature learning method for sign language recognition
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2024.112288.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Sign language recognition (SLR) is an effective solution to communication barriers experienced by hearing and vocally impaired individuals with other communities. Its applications extend to human-robot interaction (HRI), virtual reality (VR), and augmented reality (AR). However, the diverse nature of sign languages, stemming from varying user habits and geographical regions, poses significant challenges. To address these challenges, we propose the Skeleton-based Multi-feature Learning method (SML). This method comprises a Multi Feature Aggregation (MFA) module, designed to capture the inherent relationships between different skeleton-based features, enabling effective fusion of complementary information. Furthermore, we propose the Self knowledge distillation Guided Adaptive Residual Decoupled Graph Convolutional Network (SGARDGCN) for feature extraction. SGAR-DGCN consists of three components: a Self Knowledge Distillation (SKD) mechanism to enhance model training, convergence, and accuracy; a DGCN-Block, incorporating Decoupled GCN and Spatio Temporal Channel attention (STC) for efficient feature extraction; and an Adaptive Residual Block (ARes-Block) for cross-layer information fusion. Experimental results demonstrate that our SML method outperforms state-of-the-art approaches on the WLASL (55.85%) and AUTSL (96.85%) datasets, solely utilizing skeleton data. Code is available at https://github.com/DzwFine37/SML.
Keyword:
Skeleton
Sign language recognition
Multi-feature aggregation
Adaptive residual
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
A prospective study of surgical margin status in oral squamous cell carcinoma: A preliminary report
PREAI
分享收藏
The Written Park: Reading Multiple Urban Park Subjectivities Through Signage, Writing, and Graffiti
PREAI
分享收藏Skeleton-Based Multifeatures and Multistream Network for Real-Time Action Recognition基于骨架的多特征和多流网络的实时动作识别.png)
IEEE SENSORS JOURNAL
IEEE SENSORS JOURNALIF4.5
PREAI
分享收藏
查看更多内容