.png){kind=icon}
.png){kind=icon}
Return
Brain-Controlled Augmented Hearing for Spatially Moving Conversations in Multi-Talker Environments
.png){kind=icon}
OA
AI
DOI：10.1002/advs.202401379.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Focusing on a specific conversation amidst multiple interfering talkers is challenging, especially for those with hearing loss. Brain-controlled assistive hearing devices aim to alleviate this problem by enhancing the attended speech based on the listener's neural signals using auditory attention decoding (AAD). Departing from conventional AAD studies that relied on oversimplified scenarios with stationary talkers, a realistic AAD task that involves multiple talkers taking turns as they continuously move in space in background noise is presented. Invasive electroencephalography (iEEG) data are collected from three neurosurgical patients as they focused on one of the two moving conversations. An enhanced brain-controlled assistive hearing system that combines AAD and a binaural speaker-independent speech separation model is presented. The separation model unmixes talkers while preserving their spatial location and provides talker trajectories to the neural decoder to improve AAD accuracy. Subjective and objective evaluations show that the proposed system enhances speech intelligibility and facilitates conversation tracking while maintaining spatial cues and voice quality in challenging acoustic environments. This research demonstrates the potential of this approach in real-world scenarios and marks a significant step toward developing assistive hearing technologies that adapt to the intricate dynamics of everyday auditory experiences. This paper presents a novel brain-controlled binaural hearing device algorithm, evaluated with a realistic experimental paradigm. The method integrates binaural speech separation with auditory attention decoding to enhance the attended conversation, demonstrating substantial improvement in speech intelligibility and reduction in listening effort in dynamic, moving multi-talker acoustic environments. This research marks a significant advancement toward practical brain-controlled hearing technologies. image
Keywords:
auditory attention decoding
brain-computer interfaces
intracranial electroencephalography
hearing aids
speech separation
deep learning
psychophysics
Journal
IF:
14.1
Papers:
1.7W
Citations:
11.5W