.png){kind=icon}
返回
Learning Three-dimensional Skeleton Data from Sign Language Video
.png){kind=icon}
OA
AI
DOI：10.1145/3377552.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Data for sign language research is often difficult and costly to acquire. We therefore present a novel pipeline able to generate motion three-dimensional (3D) skeleton data from single-camera sign language videos only. First, three recurrent neural networks are learned to infer the three-dimensional position data of body, face, and finger joints for a high resolution of the signer's skeleton. Subsequently, the angular displacements of all joints over time are estimated using inverse kinematics and mapped to a virtual sign avatar for animation. Last, the generated data are evaluated in detail, including a sign language recognition and sign language synthesis scenario. Utilizing a neural word classifier trained on real motion capture data, we reliably classify word segments built from our newly generated position data with similar accuracy as motion capture data (absolute difference 3.8%). Furthermore, qualitative evaluation of sign animations shows that the avatar performs natural movements that are comprehensible and resemble animations created with original motion capture data.
Keyword:
3D pose estimation
recurrent neural networks
sign language synthesis
sign language recognition
data augmentation
期刊
IF:
6.6
论文数:
1.5K
被引数:
6.2K
机构
引用论文
.png){kind=icon}
分享收藏
查看更多内容