.png){kind=icon}
返回
Continuous sign language recognition algorithm based on object detection and variable-length coding sequence
.png){kind=icon}
OA
AI
DOI：10.1038/s41598-024-78319-0.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Currently, continuous sign language recognition faces challenges such as difficulty in acquiring skeletal data, long training time for Three-Dimensional convolutional neural networks, and easy occlusion and blurring of hands. To address these problems, this paper proposes a continuous sign language recognition method based on target detection and coding sequence. The algorithm uses Dual-branch Shuffle Attention Mechanis-You Only Look Once version X (DSA-YOLOX) detection network to detect the head and hands, and encodes the sign language video according to the partition to achieve the transformation from Three-Dimensional to One-Dimensional; and then uses the proposed Bi-directional Long Short-Term Memory (BiLSTM) hand coding sequence classification model with jointly weighted Fast Dynamic Time Warping (FastDTW) to extract hand coding similarity and features while reducing the number of parameters to achieve the classification and recognition of unequal-length hand coding sequences. From the results of ablation experiments and comparison experiments, all parts of the improvement perform well. The word error rate (WER) of this paper's method is reduced by 21.26% compared to Dynamic Time Warping-Hidden Markov Model (DTW-HMM) and 11.53% compared to Long Short-Term Memory-A(LSTM-A); the Giga Floating-point Operations Per Second(GFLOPs) of the algorithm are reduced dramatically, which is about 1/13 of the Visual Alignment Constraint(VAC) model and 1/57 of the Spatial-Temporal Multi-Cue(STMC) model; and the algorithm takes better account of the speed and accuracy of sign language recognition.
Keyword:
Continuous sign language recognition
Image partitioning and coding
Unequal-length time sequence
Weighted FastDTW
BiLSTM
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.9
论文数:
27.9W
被引数:
83.5W
机构
暂无机构信息
引用论文
分享收藏Sign Language Recognition Using Graph and General Deep Neural Network Based on Large Scale Dataset基于大规模数据集的图和通用深度神经网络手语识别.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏Generation of hydroxyl radicals by urban suspended particulate air matter. The role of iron ions城市悬浮颗粒物产生羟基自由基。铁离子的作用:
PREAI
分享收藏Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks通过渐进式变换器和混合密度网络进行连续3D多通道手语制作OAAI
OAAI分享收藏
查看更多内容