Generative learning-based lightweight MRI brain tumor segmentation with missing modalities

doi:10.1016/j.eswa.2024.125478

Generative learning-based lightweight MRI brain tumor segmentation with missing modalities

2025-02-01

0

PRE

AI

DOI：10.1016/j.eswa.2024.125478

Original

Original request for help

Save

Abstract

En 中文

Accurate segmentation of brain tumors across multiple MRI modalities is crucial for clinical diagnosis and prognosis. However, encountering missing modalities due to practical factors is common. Recent years have witnessed the emergence of various approaches, including 3D convolutional and transformer-based models, showcasing impressive performance in addressing this challenge. Nonetheless, these methods often suffer from high GPU memory requirements during inference, limiting their applicability. To explore alternative solutions, we turned to 2D-convolution-based techniques. Generative learning-based methods have produced visually appealing reconstructed MRI images, yet few consider the gap between visualization and segmentation. Drawing inspiration from clinicians who leverage cross-modality information for tumor estimation, we adopt a generative modality reconstruction task for pre-training, followed by fine-tuning for segmentation on MRI data with missing modalities. Furthermore, our investigation revealed common feature representations shared between reconstruction and segmentation tasks in latent space, as demonstrated through feature distributions. By utilizing a multi-encoder Unet architecture with fine-tuned adjustments, our method not only attains competitive performance in segmentation but also markedly decreases GPU memory usage during inference.

Keywords:

Brain tumor segmentation

Missing modalities

Multimodal segmentation

MRI

Generative learning-based lightweight MRI brain tumor segmentation with missing modalities

Abstract

Journal

Organization