.png){kind=icon}
返回
An fMRI-based visual decoding framework combined with two-stage learning and asynchronous iterative update strategy
.png){kind=icon}
PRE
AI
DOI：10.1007/s10044-024-01347-z.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Reconstructing visual stimulus information from evoked brain activity is a significant task of visual decoding of the human brain. However, due to the limited size of published functional magnetic resonance imaging (fMRI) datasets, it is difficult to adequately train a complex network with a large number of parameters. Furthermore, the dimensions of fMRI data in existing datasets are extremely high, and the signal-to-noise ratio of the data is relatively low. To address these issues, we design an fMRI-based visual decoding framework that incorporates additional self-supervised training on the encoder and decoder to alleviate the problem of insufficient model training due to limited datasets. Furthermore, we propose a iterative Two Part and Two Stage learning method involving a teacher (supervised)-student (self-supervised) setup and encoder-decoder asynchronous updates strategy. This approach allows the encoder and decoder to mutually reinforce and iteratively update each other under the guidance of a teacher model. The analysis of the ablation experiments demonstrates that the proposed framework can effectively improve the reconstruction accuracy. The experimental results show that the proposed method achieves better visual reconstruction from evoked brain activity of the human brain and that its reconstruction accuracy is superior to that of existing methods.
Keyword:
Asynchronous iterative update strategy
Brain activity
fMRI
Two-stage learning
期刊
IF:
2
论文数:
1.9K
被引数:
1.9K
机构
暂无机构信息
引用论文
Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders.png)
NEURON
OAAI
NEURONIF15
OAAI分享收藏Decoding natural images from evoked brain activities using encoding models with invertible mappingNEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏Generic decoding of seen and imagined objects using hierarchical visual featuresNATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏分享收藏
查看更多内容