Multi-label, multi-task CNN approach for context-based emotion recognition

doi:10.1016/j.inffus.2020.11.007

Multi-label, multi-task CNN approach for context-based emotion recognition

2021-12-01

34

PRE

AI

I

Ilyes Bendjoudi *

F

Frédéric Vanderhaegen

DOI：10.1016/j.inffus.2020.11.007

Original

Original request for help

Save

Abstract

En 中文

This paper proposes a new deep learning architecture for context-based multi-label multi-task emotion recognition. The architecture is built from three main modules: (1) a body features extraction module, which is a pre-trained Xception network, (2) a scene features extraction module, based on a modified VGG16 network, and (3) a fusion-decision module. Moreover, three categorical and three continuous loss functions are compared in order to point out the importance of the synergy between loss functions when it comes to multi-task learning. Then, we propose a new loss function, the multi-label focal loss (MFL), based on the focal loss to deal with imbalanced data. Experimental results on EMOTIC dataset show that MFL with the Huber loss gave better results than any other combination and outperformed the current state of art on the less frequent labels.

Keywords:

Emotion recognition

Loss function

Multi-task machine learning

Deep learning

Unbalanced data

Multi-label, multi-task CNN approach for context-based emotion recognition

Abstract

AI Summary

Journal

Organization