.png){kind=icon}
返回
Self-supervised cross-modal visual retrieval from brain activities
.png){kind=icon}
OA
AI
DOI：10.1016/j.patcog.2023.109915.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We study the problem of restoring visual stimuli from visually-evoked electroencephalography (EEG) signals. Using a supervised classification-then-generation framework, the reconstruction-based approaches learn the mapping between distributions of two modalities but fail to reproduce the exact visual stimulus. Instead, we propose a self-supervised cross-modal retrieval paradigm that seeks instance-level alignment by maximizing the mutual information between the EEG encoding and associated visual stimulus. We demonstrate the threefold advantages of the self-supervised retrieval over supervised reconstruction on the largest visual-evoked EEG dataset with two evaluation protocols. First, it restores the exact visual stimulus without accessing the image class information, which was not possible with previous approaches. Second, it produces more recognizable results than generated ones and bypasses the challenge of training an image generator. Finally, it illustrates the benefits of self-supervision over supervised models in handling open-set data.
Keyword:
Visual stimuli recovery
Cross-modal retrieval
Self-supervised learning
Brain-Computer Interface
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Blinded clinical evaluation of positron emission tomography for diagnosis of probable Alzheimer's disease.png)
Neurology
NeurologyIF0
PREAI
分享收藏BrainPrint: EEG biometric identification based on analyzing brain connectivity graphsBrainPrint: 基于分析大脑连接图的EEG生物特征识别PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏
查看更多内容