.png){kind=icon}
返回
Dialocalization: Acoustic Speaker Diarization and Visual Localization as Joint Optimization Problem
.png){kind=icon}
PRE
AI
DOI：10.1145/1865106.1865111.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The following article presents a novel audio-visual approach for unsupervised speaker localization in both time and space and systematically analyzes its unique properties. Using recordings from a single, low-resolution room overview camera and a single far-field microphone, a state-of-the-art audio-only speaker diarization system (speaker localization in time) is extended so that both acoustic and visual models are estimated as part of a joint unsupervised optimization problem. The speaker diarization system first automatically determines the speech regions and estimates who spoke when, then, in a second step, the visual models are used to infer the location of the speakers in the video. We call this process dialocalization. The experiments were performed on real-world meetings using 4.5 hours of the publicly available AMI meeting corpus. The proposed system is able to exploit audio-visual integration to not only improve the accuracy of a state-of-the-art (audio-only) speaker diarization, but also adds visual speaker localization at little incremental engineering and computation costs. The combined algorithm has different properties, such as increased robustness, that cannot be observed in algorithms based on single modalities. The article describes the algorithm, presents benchmarking results, explains its properties, and systematically discusses the contributions of each modality.
Keyword:
Experimentation
Speech
visual localization
speaker diarization
multimodal integration
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
引用论文
.png){kind=icon}
Speaker diarization for multiple-distant-microphone meetings using several sources of information使用多个信息源进行多个远程麦克风会议的扬声器diarization
OAAI
OAAI分享收藏
分享收藏分享收藏
查看更多内容