.png){kind=icon}
返回
Implicit Memory-Based Variational Motion Talking Face Generation
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2024.3356415.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Speech-driven facial animation is a challenging problem where each input audio can have multiple plausible facial outputs, leading to overly smooth results. Although the two-stage framework of audio-to-motion model and neural rendering models can partially mitigate this issue, it lacks crucial details like emotions and wrinkles. To overcome these limitations, we introduce a variational motion generator with implicit memory. By incorporating implicit memory into the audio-to-motion model, we capture high-level semantics in the shared latent space of audio expressions, resulting in accurate and expressive facial landmark generation. Next, we introduce attention with time bias to effectively maintain the consistency of audio motion and adopt a periodic position encoding strategy to provide summarization capability for longer audio sequences. Experimental results demonstrate that our approach outperforms previous methods, yielding more extensive and realistic speech-driven facial animation.
Keyword:
Implicit memory
speech-driven facial
audio-to-motion
期刊
IF:
9.6
论文数:
1.1W
被引数:
1.9W
机构
暂无机构信息
引用论文
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units休伯特: 通过隐藏单元的掩蔽预测进行自我监督的语音表示学习
OAAI
OAAI分享收藏A HEMATOLOGICAL APPROACH TO BASQUE ISOLATION IN TWO FRENCH BASQUE VILLAGES*对法国巴斯克地区两个巴斯克村庄隔离的血液学方法*
PREAI
分享收藏Assessment of best single sample for finding chlamydia in women with and without symptoms: a diagnostic test study.png)
BMJ
OAAI
BMJIF0
OAAI分享收藏
查看更多内容