.png){kind=icon}
.png){kind=icon}
Return
Visual-Guided Dual-Spatial Interaction Network for Fine-Grained Brain Semantic Decoding
.png){kind=icon}
PRE
AI
DOI：10.1109/TIM.2024.3480232.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Brain semantic decoding has received a surge of attention in the computer vision and neuroscience disciplines. However, existing techniques ignore the sparse and implicit semantic analysis issue of the brain signal, leading to coarse-grained brain semantic decoding. In this article, a visual-guided dual-spatial interaction network (VDIN) is first proposed to significantly facilitate the fine-grained brain semantic decoding, with the multiway guidance of the visual message (text and image). Specifically, the local dual-spatial interaction operation is proposed to exploit the explicit-implicit coupling semantic cues between brain and text, via the text-aware space. Moreover, the above operation simultaneously attends to the multilevel semantic cues between brain and image from the more high-resolution image-aware space. Furthermore, the global dual-spatial interaction operation is utilized to integrate and modify the obtained local semantic cues into the global one. This naturally boosts the investigation of visual-brain consistency and complementary, leading to the more expressive and fine-grained brain semantic decoding architecture. For evaluation, VDIN was verified on the public brain-visual benchmark EEGCVPR40, and the experiments demonstrated that the proposed model outperforms the best baseline with an improvement of 15.97%. Note that our model still achieves competitive performance even considering the brain-spatial space associated with fewer channels.
Keywords:
Brain semantic decoding
dual-spatial inter- action
image-aware semantic analysis
text-aware semantic analysis
text-aware semantic analysis
visual-guided analysis
visual-guided analysis
visual-guided analysis
Journal
IF:
5.9
Papers:
1.9W
Citations:
5.8W