.png){kind=icon}
Return
Graph-based social relation inference with multi-level conditional attention
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2024.106216.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Social relation inference intrinsically requires high-level semantic understanding. In order to accurately infer relations of persons in images, one needs not only to understand scenes and objects in images, but also to adaptively attend to important clues. Unlike prior works of classifying social relations using attention on detected objects, we propose a MUlti-level Conditional Attention (MUCA) mechanism for social relation inference, which attends to scenes, objects and human interactions based on each person pair. Then, we develop a transformer -style network to achieve the MUCA mechanism. The novel network named as Graphbased Relation Inference Transformer (i.e., GRIT) consists of two modules, i.e., a Conditional Query Module (CQM) and a Relation Attention Module (RAM). Specifically, we design a graph -based CQM to generate informative relation queries for all person pairs, which fuses local features and global context for each person pair. Moreover, we fully take advantage of transformer -style networks in RAM for multi -level attentions in classifying social relations. To our best knowledge, GRIT is the first for inferring social relations with multilevel conditional attention. GRIT is end -to -end trainable and significantly outperforms existing methods on two benchmark datasets, e.g., with performance improvement of 7.8% on PIPA and 9.6% on PISC.
Keywords:
Social relation inference
Multi-level conditional attention
Transformer
Journal
IF:
6.3
Papers:
8.3K
Citations:
3.0W
Organization
Cited Papers
Estimation of surface roughness for turning operations using image texture features.png)
Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture
Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering ManufactureIF0
PREAI
ShareSaveShareSaveShareSaveTemperature effect on crystal structures, morphological shapes, and functional groups of zinc oxide
OAAI
OAAIShareSave
View more