Multi-scale features based interpersonal relation recognition using higher-order graph neural network

doi:10.1016/j.neucom.2021.05.097

Multi-scale features based interpersonal relation recognition using higher-order graph neural network

2021-10-01

17

PRE

AI

DOI：10.1016/j.neucom.2021.05.097

Original

Original request for help

Save

Abstract

En 中文

Interpersonal relation plays an essential role to gain understandings on how people interact with each other. In computer vision, interpersonal relations provide vital information to interpret people's behaviors. However, the existing research has either omitted the interaction information between subjects or the structural information in the images. In this paper, we propose a new architecture to reason interpersonal relations based on higher-order graph networks and multi-scale features. First, we extract features of the whole images, the facial features, and the union region of face pairs. Apart from the pixel wise features, we also consider the positional features of face-to-face pairs and the spatial scene cues. Higher-order Graph Neural Networks (GNNs) were employed to map out the interpersonal relations based on the feature extracted. Experimental results show that the proposed Higher-order Graph Neural Networks with multi-scale features can effectively recognize the social relations in images with over 5% improvement in absolute balanced accuracy compared with the state-of-the-art work. (c) 2021 Elsevier B.V. All rights reserved.

Keywords:

Social relation recognition

Higher-order graph neural network

Multi-scale features

Graph reasoning

Interpersonal relation reasoning

Multi-scale features based interpersonal relation recognition using higher-order graph neural network

Abstract

AI Summary

Journal

Organization