.png){kind=icon}
返回
Expression-Preserving Face Frontalization Improves Visually Assisted Speech Processing
.png){kind=icon}
OA
AI
DOI：10.1007/s11263-022-01742-1.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Face frontalization consists of synthesizing a frontal view from a profile one. This paper proposes a frontalization method that preserves non-rigid facial deformations, i.e. facial expressions. It is shown that expression-preserving frontalization boosts the performance of visually assisted speech processing. The method alternates between the estimation of (i) the rigid transformation (scale, rotation, and translation) and (ii) the non-rigid deformation between an arbitrarily-viewed face and a face model. The method has two important merits: it can deal with non-Gaussian errors in the data and it incorporates a dynamical face deformation model. For that purpose, we use the Student's t-distribution in combination with a Bayesian filter in order to account for both rigid head motions and time-varying facial deformations, e.g. caused by speech production. The zero-mean normalized cross-correlation score is used to evaluate the ability of the method to preserve facial expressions. The method is thoroughly evaluated and compared with several state of the art methods, either based on traditional geometric models or on deep learning. Moreover, we show that the method, when incorporated into speech processing pipelines, improves word recognition rates and speech intelligibility scores by a considerable margin.
Keyword:
Face frontalization
Student's t-distribution
Robust point registration
Bayesian filtering
Lip reading
Audio-visual speech enhancement
Variational auto-encoders
期刊
IF:
9.3
论文数:
3.9K
被引数:
2.8W
机构
引用论文
.png){kind=icon}
Quality of life and audiologic performance through the ability to phone of cochlear implant users
PREAI
分享收藏
查看更多内容