.png){kind=icon}
.png){kind=icon}
Return
Embedding-Efficient Brain-to-Image Reconstruction Using Diffusion Models
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-12478-4_10.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Visual brain decoding is the process of reconstructing the images a person sees by analyzing their brain activity, as measured using functional MRI, electroencephalography, or magnetoencephalography. Recent advances in generative AI, particularly diffusion models, have significantly improved the semantic quality of the reconstructed images. In this work, we investigate the potential for improving the computational efficiency of brain-to-image models by focusing on a reduced number of informative embeddings. Specifically, we explore whether high-quality image reconstructions can be achieved by predicting a small, selected subset of CLIP-derived embeddings. We perform evaluations using the Natural Scenes Dataset, applying both a simple single-subject ridge regression model and a multi-subject federated learning framework adapted from a recent approach. The results demonstrate that embedding-efficient decoding can achieve competitive performance while substantially reducing model size, although with a decrease in quality on certain metrics. Code available at: https://github.com/IoanaGabor/efficient-vbd.
Keywords:
Braincomputer interfaces
functional magnetic resonance imaging (fMRI)
image reconstruction
diffusion models
neural decoding
CLIP embeddings
federated learning
ridge regression
Journal
I
IF:
0
Papers:
23
Citations:
0