.png){kind=icon}
Return
EmoComicNet: A multi-task model for comic emotion recognition
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110261.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The emotion and sentiment associated with comic scenes can provide potential information for inferring the context of comic stories, which is an essential pre -requisite for developing comics' automatic content understanding tools. Here, we address this open area of comic research by exploiting the multi -modal nature of comics. The general assumptions for multi -modal sentiment analysis methods are that both image and text modalities are always present at the test phase. However, this assumption is not always satisfied for comics since comic characters' facial expressions, gestures, etc., are not always clearly visible. Also, the dialogues between comic characters are often challenging to comprehend the underlying context. To deal with these constraints of comic emotion analysis, we propose a multi -task -based framework, namely EmoComicNet, to fuse multi -modal information (i.e., both image and text) if it is available. However, the proposed EmoComicNet is designed to perform even when any modality is weak or completely missing. The proposed method potentially improves the overall performance. Besides, EmoComicNet can also deal with the problem of weak or absent modality during the training phase.
Keywords:
Comic analysis
Multi-modal emotion recognition
Document image processing
Deep learning
Multi-task learning
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
No organization information available
Cited Papers
ShareSave
Childhood Fever: Correlation of Diagnosis With Temperature Response to Acetaminophen.png)
Pediatrics
PediatricsIF0
PREAI
ShareSaveShareSaveShareSave
View more