.png){kind=icon}
Return
Multimodal Emotion Recognition Using Modality-Wise Knowledge Distillation
.png){kind=icon}
OA
AI
DOI：10.3390/s25206341.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal emotion recognition (MER) aims to estimate emotional states utilizing multiple sensors simultaneously. Most previous MER models extract unimodal representation via modality-wise encoders and combine them into a multimodal representation to classify the emotion, and these models are trained with an objective for the final output of the MER. If an encoder for a specific modality is optimized better than others at some point of the training procedure, the parameters for the other encoders may not be sufficiently updated to provide optimal performance. In this paper, we propose a MER using modality-wise knowledge distillation, which adapts the unimodal encoders using pre-trained unimodal emotion recognition models. Experimental results on CREMA-D and IEMOCAP databases demonstrated that the proposed method outperformed previous approaches to overcome the optimization imbalance phenomenon and could also be combined with these approaches effectively.
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
No journal information available
Organization
No organization information available
Cited Papers
ShareSaveA Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations
OAAI
OAAIShareSaveShareSaveShareSaveSpeech Emotion Recognition Incorporating Relative Difficulty and Labeling Reliability.png)
SENSORSOAAI
SENSORSIF3.5
OAAIShareSaveEmotion Recognition in Conversations: A Survey Focusing on Context, Speaker Dependencies, and Fusion MethodsElectronicsOAAI
ElectronicsIF0
OAAIShareSaveA multimodal fusion emotion recognition method based on multitask learning and attention mechanismNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSave
View more