.png){kind=icon}
.png){kind=icon}
Return
MFSleepNet: An Interactive Multimodal Fusion Framework for Automatic Sleep Staging
.png){kind=icon}
OA
AI
DOI：10.3390/s26103085.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Accurate automatic sleep staging remains challenging due to complex temporal dynamics, inter-subject variability, and the difficulty of effectively integrating heterogeneous physiological signals. Electroencephalogram (EEG) and electrooculogram (EOG) recordings provide complementary information for sleep analysis; however, most existing multimodal approaches rely on simple feature concatenation, which limits their ability to capture structured inter-modality relationships. This paper proposes MFSleepNet, a multimodal sleep staging framework that explicitly models interactions between EEG and EOG signals. The proposed system incorporates a multimodal feature fusion module to enable bidirectional information exchange between modality-specific representations, followed by a gated temporal-channel attention mechanism to adaptively emphasize informative temporal segments and signal channels, facilitating joint representation learning while preserving modality-specific characteristics. Experiments on three public datasets (Sleep-EDF, SHHS, and HSP) under an epoch-level cross-validation protocol show that MFSleepNet consistently outperforms representative single-modality and multimodal baseline methods in terms of overall accuracy, Cohen’s κ, and Macro-F1. Ablation studies further demonstrate the contribution of each functional module. Correlation analysis indicates stage-dependent variations in EEG–EOG relationships, while interaction-based experiments show that explicit feature interaction improves both joint and modality-specific representations. Grad-CAM visualizations provide interpretability of model decisions. External validation on unseen subjects reveals a noticeable performance drop, highlighting the challenges of inter-subject variability and the limited baseline generalization capability of the model. To address this, a lightweight subject-specific adaptation strategy is introduced, which improves performance using a small amount of labeled subject-specific data. Overall, the proposed framework provides an effective and interpretable solution for multimodal sleep staging while emphasizing the importance of structured inter-modality interaction and subject-adaptive modeling in practical applications.
Keywords:
sleep staging
physiological signal processing
EEG and EOG fusion
multimodal learning
subject-specific adaptation
Journal
IF:
3.5
Papers:
7.1W
Citations:
20.9W