EEG-Based Multimodal Emotion Recognition: A Machine Learning Perspective

Huan Liu

doi:10.1109/TIM.2024.3369130

EEG-Based Multimodal Emotion Recognition: A Machine Learning Perspective

2024-01-01

10

PRE

AI

DOI：10.1109/TIM.2024.3369130

原文链接

摘要

En 中文

Emotion, a fundamental trait of human beings, plays a pivotal role in shaping aspects of our lives, including our cognitive and perceptual abilities. Hence, emotion recognition also is central to human communication, decision-making, learning, and other activities. Emotion recognition from electroencephalography (EEG) signals has garnered substantial attention due to advantages such as noninvasiveness, high speed, and high temporal resolution; driven also by the complementarity between EEG and other physiological signals at revealing emotions, recent years have seen a surge in proposals for EEG-based multimodal emotion recognition (EMER). In short, EEG-based emotion recognition is a promising technology in medical measurements and health monitoring. While reviews exist, which explore emotion recognition from multimodal physiological signals, they focus mostly on general combinations of modalities and do not emphasize studies that center on EEG as the fundamental modality. Furthermore, existing reviews take a methodology-agnostic perspective, primarily concentrating on the biomedical basis or experimental paradigms, thereby giving little attention to the methodological characteristics unique to this field. To address these gaps, we present a comprehensive review of current EMER studies, with a focus on multimodal machine learning models. The review is structured around three key aspects: multimodal feature representation learning, multimodal physiological signal fusion, and incomplete multimodal learning models. In doing so, the review sheds light on the advances and challenges in the field of EMER, thus offering researchers who are new to the field a holistic understanding. The review also aims to provide valuable insight that may guide new research in this exciting and rapidly evolving field.

Keyword:

Electroencephalography

Emotion recognition

Physiology

Electromyography

Electrooculography

Reviews

Biomedical monitoring

Electroencephalography (EEG)

emotion recognition

machine learning

multimodal learning

multimodal physiological signal

EEG-Based Multimodal Emotion Recognition: A Machine Learning Perspective

摘要

期刊

机构

引用论文