.png){kind=icon}
返回
Improving speech embedding using crossmodal transfer learning with audio-visual data
.png){kind=icon}
OA
AI
DOI：10.1007/s11042-018-6992-3.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Learning a discriminative voice embedding allows speaker turns to be compared directly and efficiently, which is crucial for tasks such as diarization and verification. This paper investigates several transfer learning approaches to improve a voice embedding using knowledge transferred from a face representation. The main idea of our crossmodal approaches is to constrain the target voice embedding space to share latent attributes with the source face embedding space. The shared latent attributes can be formalized as geometric properties or distribution characterics between these embedding spaces. We propose four transfer learning approaches belonging to two categories: the first category relies on the structure of the source face embedding space to regularize at different granularities the speaker turn embedding space. The second category -a domain adaptation approach- improves the embedding space of speaker turns by applying a maximum mean discrepancy loss to minimize the disparity between the distributions of the embedded features. Experiments are conducted on TV news datasets, REPERE and ETAPE, to demonstrate our methods. Quantitative results in verification and clustering tasks show promising improvement, especially in cases where speaker turns are short or the training data size is limited. The analysis also gives insights the embedding spaces and shows their potential applications.
Keyword:
Speaker diariazation
Multimodal identification
Metric learning
Transfer learning
Deep learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3
论文数:
2.0W
被引数:
3.2W
机构
引用论文
Regional lymphatic recurrence after salvage surgery for ipsilateral breast tumor recurrence of breast cancer without local treatment for regional lymphatic basin乳腺癌同侧乳腺肿瘤复发行挽救手术后，未行区域淋巴引流区局部治疗时的区域淋巴复发。
PREAI
分享收藏Experimental Investigation of Low-Frequency Distributed Acoustic Strain-Rate Responses to Propagating Fractures.png)
SPE Journal
SPE JournalIF0
PREAI
分享收藏Abundance matters: a field experiment testing the more individuals hypothesis for richness–productivity relationshipsOecologia
OAAI
OecologiaIF0
OAAI分享收藏分享收藏MYCN Fish combined with immunohistochemistry is a more prognostic factor for neuroblastomaMYCN Fish联合免疫组化是神经母细胞瘤更具有预后价值的因素。OAAI
IF0
OAAI分享收藏
查看更多内容