.png){kind=icon}
返回
Audio-Visual Based Online Multi-Source Separation
.png){kind=icon}
PRE
AI
DOI：10.1109/TASLP.2022.3156758.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Meeting or conference assistance is a popular application that typically requires compact configurations of co-located audio and visual sensors. This paper proposes a novel solution for online separation of an unknown and time-varying number of moving sources using only a single microphone array co-located with a single visual device. The approach exploits the complementary nature of simultaneous audio and visual measurements, accomplished by a model-centric 3-stage process of detection, tracking, and (spatial) filtering, which performs separation in a block-wise or recursive fashion. Fusing the measurements requires solving the multi-modal space-time permutation problem, since the audio and visual measurements reside in different observation spaces, but also are unidentified or unlabeled (with respect to the unknown and time-varying number of sources), and are subject to noise, extraneous measurements and missing measurements. A labeled random finite set tracking filter is applied to resolve the permutation problem and recursively estimate the source identities and trajectories. A time-varying set of generalized side-lobe cancellers is constructed based on the tracking estimates to perform online separation. Evaluations are undertaken with live human speakers.
Keyword:
Visualization
Noise measurement
Sensors
Radio frequency
Position measurement
Music
Training
Audio-visual
source separation
spatial filtering
labeled random finite sets
generalized labeled multi-Bernoulli
期刊
I
IF:
5.1
论文数:
2.6K
被引数:
1.1W
机构
引用论文
.png){kind=icon}
查看更多内容