.png){kind=icon}
返回
CoFaCo: Controllable Generative Talking Face Video Coding
.png){kind=icon}
PRE
AI
DOI：10.1109/TIP.2025.3648562.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Efficient talking face video coding and control are crucial in modern video communication, reshaping how individuals connect, collaborate, and interact. Coding seeks to reduce transmission costs, while control enables the realization of user-customizable facial expressions and head poses in the transmitted videos. However, the compression efficiency of the common par-adigm of applying control algorithms before video coding is not satisfactory. In this paper, we propose an efficient, Controllable Generative Talking Face Video Coding (CoFaCo) framework, wh-ich seamlessly integrates control into the coding process. Specific-ally, CoFaCo projects talking face videos into ultra-compact and semantic feature representations that can be customized by users before compression. To enable independent controls of pose and expression, we design a set of sophisticated losses to accurately de-couple the pose and expression direction codes. Once the decoupled direction codes and the semantic face representations are obtained, the pose and expression control modules can be effectively learned to generate decoupled, controlled pose and expression direction codes. The controlled direction codes are subsequently smoothed to enhance temporal consistency in the controlled video output by the generators. Experimental results demonstrate that CoFaCo achieves competitive compression efficiency in ultra-low bit rate video reconstruction and control tasks, providing valuab-le insights for advancing face video communication with diverse control capabilities.
Keyword:
Generative neural network
talking face video coding
customizable control
face representation
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
.png){kind=icon}
分享收藏
分享收藏
Learning Adaptive Spatial Coherent Correlations for Speech-Preserving Facial Expression Manipulation学习自适应空间相干相关性，用于语音保留的面部表情操控
PREAI
分享收藏Stochastic Latent Talking Face Generation Toward Emotional Expressions and Head Poses面向情感表达和头部姿态的随机潜在 talking face 生成
PREAI
分享收藏
查看更多内容