.png){kind=icon}
返回
Speech-Driven Gesture Generation Using Transformer-Based Denoising Diffusion Probabilistic Models
.png){kind=icon}
OA
AI
DOI：10.1109/THMS.2024.3456085.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
While it is crucial for human-like avatars to perform co-speech gestures, existing approaches struggle to generate natural and realistic movements. In the present study, a novel transformer-based denoising diffusion model is proposed to generate co-speech gestures. Moreover, we introduce a practical sampling trick for diffusion models to maintain the continuity between the generated motion segments while improving the within-segment motion likelihood and naturalness. Our model can be used for online generation since it generates gestures for a short segment of speech, e.g., 2 s. We evaluate our model on two large-scale speech-gesture datasets with finger movements using objective measurements and a user study, showing that our model outperforms all other baselines. Our user study is based on the Metahuman platform in the Unreal Engine, a popular tool for creating human-like avatars and motions.
Keyword:
Diffusion models
Data models
Transformers
Feature extraction
Noise reduction
Avatars
Skeleton
Robots
Motion segmentation
Motion capture
Co-speech gesture
deep learning
gesture-based interaction
social interaction
期刊
IF:
4.4
论文数:
1.1K
被引数:
3.5K
机构
引用论文
.png){kind=icon}
查看更多内容