.png){kind=icon}
.png){kind=icon}
Return
Hypercomplex context guided interaction modeling for scene graph generation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109634.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Intuitively, humans can consciously and subjectively attend to the interactions between objects, and thus infer reasonable visual relations. However, mainstream approaches of Scene Graph Generation (SGG) strive to alleviate the long-tailed distribution problem with various complicated re-weighting strategies, where a simple concatenation of the refined object features is treated as the final representation of visual relations. In spite of their remarkable progress, such an operation overlooks the importance of interac-tion on relation recognition. To tackle the problem, this work devises a hyperComplex-Context guided Interaction Modeling (CCIM for short) plug-in, which can be successfully assimilated by the existing methods for performance improvement. Specifically, we first extract the contextual relation feature determined by the constraint relation ti union(head, tail) - head object - tail object. Then, we encode the features of re-lations and objects into hypercomplex space, with three imaginary components, to learn more expressive representations for SGG. Next, guided by the context, we can capture the interaction between a head or tail object and their relation through the Hamilton product. We further reinforce the interaction between enhanced hypercomplex-valued representations of the two entities with Quaternion inner product. At last, the concatenation of all components from the learned hypercomplex feature is adopted as our final relation representation. Extensive experiments on the popular benchmark Visual Genome in various exist-ing approaches demonstrate the effectiveness and generalization of our proposed model-agnostic method under comprehensive evaluation metrics.(c) 2023 Published by Elsevier Ltd.
Keywords:
Scene graph generation
Context guidence
Interaction modeling
Hypercomplex embedding