.png){kind=icon}
返回
MSDA-Net: Multi-source Domain Adaptive Network for Multi-modal Emotion Recognition
.png){kind=icon}
PRE
AI
DOI：10.1145/3786588.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Electroencephalogram (EEG) has shown g reat potential in multi-modal emotion recognition (MER) due to its ability to directly capture emotional states. However, the nonstationarity of EEG signals leads to significant variations across subjects and sessions, posing challenges for subject-independent MER. While previous methods have made significant progress, they often fail to integrate multimodal signals into transfer learning frameworks effectively. To address this limitation, we propose a Multi-source Domain Adaptive Network (MSDA-Net) for MER, designed to mitigate cross-subject and cross-session distribution shifts and enhance recognition performance. Specifically, we first design a feature alignment module to integrate features from different modalities, generating cross-modal feature representations and extracting representative shared features. To further improve generalization, we incorporate domain-specific feature extractors to capture domain-invariant emotional representations. Additionally, we introduce an adapter module to adjust the feature representations between different modalities, aiming to capture inter-individual differences and cross-modal correlations better. Finally, we unify classification loss, discrepancy loss, and maximum mean discrepancy (MMD) loss into a joint optimization framework. Abundant experiments on the SEED and SEED-IV datasets demonstrate the superiority of MSDA-Net, highlighting its effectiveness in improving MER performance.
Keyword:
Multi-modal emotion recognition
transfer learning
electroencephalogram
domain adaptive
期刊
机构
引用论文
M3D: Manifold-Based Domain Adaptation With Dynamic Distribution for Non-Deep Transfer Learning in Cross-Subject and Cross-Session EEG-Based Emotion RecognitionM3D：基于流形的动态分布领域自适应，用于跨主体和跨会话基于EEG的情绪识别的非深度迁移学习
PREAI
分享收藏Progressive low-rank subspace alignment based on semi-supervised joint domain adaption for personalized emotion recognition.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏Multisource Associate Domain Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition
PREAI
分享收藏MPED: A Multi-Model Physiological Emotion Database for Discrete Emotion RecongnitionIEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏S. Y. Dharia, C. E. Valderrama, S. G. Camorlinga, Multimodal deep learning model for subject-independent eeg-based emotion recognition, in: 2023 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), 2023, pp. 105–110. doi:10.1109/CCECE58730.2023.10289007.S. Y. Dharia, C. E. Valderrama, S. G. Camorlinga, 基于EEG的多模态深度学习模型用于受试者无关的情绪识别，载于：2023年IEEE加拿大电气与计算机工程会议（CCECE），2023年，第105–110页。doi:10.1109/CCECE58730.2023.10289007.
IF0
PREAI
分享收藏Fusing Frequency-Domain Features and Brain Connectivity Features for Cross-Subject Emotion Recognition融合频域特征和脑连接特征的跨主体情感识别
PREAI
分享收藏
查看更多内容